I would be worried too.
What if you took a different approach and tried to be nice to him
and get him to talk about the things he doesn't like, etc...
Maybe talk to him and tell him that you get the sense that he doesn't like your pets, and that since you are sort of neighbors, you really want to work through that, not that he has to love your pets, but that you want him to understand how much YOU love and care for your pets, try and get him to respect YOUR feelings towards your animals and how YOU want to be friendly neighbors, etc....IDK - may be worth a try
