I agree what others are telling you seems pretty accurate to me.
I am not certain
why you chose to get a breed that all the breeders you talked to were basically advising
against or why a breeder
after advising against (if this is indeed the case) would sell you one, but perhaps I do not need to understand since it is your choice and I would reserve the same rights for myself.
Having said that...lol,
if you intend to keep the Yorkie and are just
looking for ways to bring dog and kids together, with Yorkies being 'human toddler-like' in my experience, maybe you could try some way simpler stuff...like having the children
treat the dog like another child...not like a dog. Skip the dog games for now, and do stuff like allow the dog to sit wherever he wants...his bed or crate, even, and have the older one's read books to
the other children including the dog, even include the dog in 'tea' parties, etc., maybe even dress-up parties if/when the dog wants to dress up...and he may need time to become comfortable before this happens, have them
TALK to the dog as if he is a PERSON rather than a dog...stuff like that. Understand you may have to supervise for the safety of all.
Also, give your children, including your Yorkie, safe dog snacks like carrot sticks at the same time...
include him into your family as a child, since Yorkies seem to think of themselves as 'humans with benefits'...like human servants, etc...at least mine do. lol If you are giving your children something toxic for the dog...like chocolate or grapes/raisins, put the dog in a safe place until all 'crumbs' are cleaned away. It is not an accident any other way if something bad happens but it will likely be expensive...or sad...maybe both...and the guilt will be enormous. Please do not take offense...it is merely fair that you know in advance of the possibility of such dangers.
You will have to be careful what foods the children eat around the dog...this
cannot be over-emphasized. Once the dog is socialized to your family, gently train him
not to snatch/grab/swallow whatever hits the floor...I would wait to start this training until he bonds and
knows you...knows you are not being mean with commands, and treat...treat...treat when he 'leave(s) it' but nothing negative when he doesn't! A cue for when he is being given food (I used "all yours" when training my dogs while my children were young) would be good as well...
reserve the cue word for yourself and spouse and
do not train for this in front of the children or they could end up giving him something toxic...like chocolate...
with the cue word.
I would hope that you have child-proofed so that poisons, cleansers, etc. are inaccessible to them and to the dog...right...and electric cords, etc., but little dogs are like cats and infants...they go down real FAST !

and so must be treated the same way.
All things take time, and this will be no exception.
Congratulations, good luck, be patient, be persistent, be gentle and kind, have fun, love, hold and cuddle, and your Yorkie baby may surprise you beyond your wildest dreams.