I agree with the member who called this man a tool. That is absolutely horrible what he said to you. I can't believe (well...yes I can) that perfect strangers can just walk up to another human being and verbally abuse them like that - shameful.
Fact is, YOU know better, WE know better, and that man knows better. It sounds to me like he is purposely trying to instigate an argument.
I'm so sorry that happened to you.