Honey, I would not worry about what people say. I have had people tell me that my Max (who is an even 6lbs) is a "big" Yorkie... my Molly is even bigger than that.... so I just don't pay any attention to them. They think that ALL Yorkies are these little itty bitty handheld toys or something I guess. Its annoying to hear someone to make a comment about how big my babies are or something else like that... but it just goes to show how much they don't know!
Be thankful that you have a bigger baby, you won't have to worry about him as much while playing, jumping, etc. My 5 month old puppy, Ginger, is 3lb 6oz and let me tell you, she scares me to death every time she jumps off the couch or is running up and down our stairs.