I had a similar problem last night! I live in a townhouse that backs to a beautiful bike trail with a mud puddle some people call a lake. But because I have nerve damage in my ankle, Crickett and I can't go down there to walk very often. So we walk around the sidewalk in front instead. (Which is more like a park). She never pottys in other peopls yards because we go BEFORE the walk. I always carry a bag tied to the leash "just in case". I do however let her smell around the sidewalk and yards. We never even stop long enough for her to potty in anyones yards!
Yesterday I waved and said hi to a neighbor as we walked by. (Normally we have a very friendly community.) The guy yelled under his breath, "(something, something)... Damn Dog, I don't even like you anyway!" WTH?!?!?!
Some people are just truly grumpy people and NOT yorkie friendly. Poor souls. I feel sorry for them. They are missing a true blessing in their lives!
Anyway when you walk past the guys ugly yard, just try to imagine how miserable and unhappy he must be instead of being mad.