In NYC, we are pretty lucky; it is a very dog friendly place. You can even bring your dog to Bergdorf's!!! I take Chewy out a lot and always bring a carrier. In my neighborhood everyone loves to see him and people ask where he is if I don't have him. I've never had a problem but do respect it when people say NO PETS.
I wonder though, do they have a right to ask what's in your bag