Since I've been working at my own business for close to ten years now, I always feel like I am forced to be nice, and the truth of it is, I'm really not that nice of a person to people I don't know,
so it's really hard for me to be calm and nice to rude people in my shop.
BUT if I'm out and about, and I think I won't run into this person anytime soon, I usually will say something, but I don't go all out. I should. Once bf did it and it was so funny and it was like a weight lifted. If you are too nice a person to not say anything, it's just going to bug you later, as it seems like it is bugging you now. Believe me, it would bug me too, and stuff like this has happened to me before. I've even shed tears over it.
Now that I am getting older, I also feel like I can say things more than biting my tongue, but of course, only in response to a rude remark, I wouldn't go up to a stranger and say something rude like that out of the blue.