I have had 3 yorkies and have had to remove all their baby teeth.
Please go to the vet and let them do it. They do a cleaning and check everything at that time also including their anals.
Usually then they will put the pup on an antibiotic so there isn't any infection and a little pain reliever for afterward.
They are only 'out of it' for the day and back to normal in a day or two.
At least they won't have the truama of you going after their mouth and make him afraid of you.