We'll put Keeley in his cage when we have some vistors (the non-dog people or little people that poke, pull and try to "play" with him).
It is fine!!! If you were harming Brownie I could see it be abuse, but if you politely leave him inside, or in his cage when you have company, big whoop!
You are fine! You should let them see why leave Brownie inside next time and they may be singing a different tune.
