The only person you need to worry about is your boss. Usually that kind of talk is generated by jealousy. In reality, if they had been doing 110% of their jobs, there may not have been a position open at the time you were hired.
When you get a break, walk over to the group and say, Hi guys. I was just feeling a bit lonely ( or bored or claustrophobic)... who are we all talking about? I normally don't engage in office gossip, it's the most useless and unproductive thing one finds in many office settings, but I couldn't resist as I already overheard most of your conversation.... I just don't know any of you and I was curious who it was....
And if they say 'You'
, just say well, it appears I'm in good company! Probably why I was hired in the first place, bc I know how to fit right in with a bunch of useless slackers and still get my job done!
Or just ignore them....