I can understand how you feel, I have 2 kids (6 and 3). I can see both sides to this and I'm the one getting turned down, but to me my Toby is a child as well and you need to take precautions to ensure his safety. Both my chidren love him and are sweet and gentle by nature. I still taught them how to pet and handle our 3lb yorkie. We have never had an accident, but my children don't terrorize him either. If Toby wants to play or cuddle he seeks them, and if they want to give him attention and he wants to nap then they let him sleep.
 
Hope you find a breeder that understands yorkies and children can co-exist