That's classic submission.
He may have been abused and is just flat out scared of men. Or he may have had the fear of God instilled in him by a man and he's trying to show he's not challenging authority.
Either way, may I suggest that the males in your family shower a little extra attention on him so he understands he has nothing to fear? Extra treats? A head ruffling once in a while? Walks?
Hopefully, he will get over it, but it may take some time. |