Not too uncommon if he gets some pieces of food trapped around those little baby chompers. Sometimes they swallow a little blood when the teeth come out and that can make breath smell a bit. I always recommend probiotics for stinky breath because usually it goes farther than the mouth. It certainly couldn't hurt in this case, but the breath should ease up when the teeth are done. Keep up with those toofers and you'll all benefit.
Probiotics: (For a yorkie, this stuff will last FOREVER!)
BacPak Plus from N'Zymes.com
For $26.00, it's about 2 years worth of no farts and comfie bellies. (Might as well split it with another yorkie parent!) I'm sure a lot of people will pay more than that for one or two sweaters that their furbaby will grow out of!
Oh also, watch out fr those breathmints. If they contain xylitol, your boy may get a nasty case of the runs!