Jackson is a bigger Yorkie at 19lbs so I definitely don't carry him around, however I bring him almost everywhere he's allowed to go on leash.
He's very well behaved in stores & loves to be with me. Any type of errands... I travel between houses a lot so he's often with me, so like little things like getting my oil changed, car washed, etc, he's always welcome at places like that. Downtown area here is very dog friendly- many stores allow dogs. Outdoor patios at restaurants... home/garden type stores, places like Home Depot, etc....
I'm very respectful. If a place specifically states no dogs, I do not bring him in. If I was ever asked to leave, I would leave respectfully. Any place that sells food I do not bring him in. Just be courteous to others, make sure your dog is socialized & trained, and be aware that not everyone likes dogs and you should be good.