I can so sympathize with your problem. It's kind of funny here in this small town there is a fairly new housing addition started, none of the houses were selling though. I wondered why and was told it's because the only access to the new addition is the same street you have to turn on to get to the towns only liquor store. None of the the Christians (which is about 90% of the town) here want to be seen turning there for fear someone driving by will think they are going to the liquor store.
I love living in my small town but the gossip can be an annoying thing, especially when you consider those doing the gossip usually have a few skeletons in their closets too.
LOL....now you know you just have to show up at a church banquet some Sunday with a big ol' roast to stop the tongues wagging!