I know how you feel - I've had to leave Thor in other people's care during potty training and it is really nerve wracking.
This is not exactly the same, but a couple years ago I had this incredibly gorgeous orchid. It was the only plant I'd ever managed to keep alive, and it grew and blossomed year after year. I gave it to a friend to plant-sit over Christmas vacation, and when I came back, it was basically kindling. And to top it off, my friend was SO proud of herself for taking care of it that I could never bring myself to tell her that it died.
Actually there was this other time when this unbelievably annoying girl decided she was going to be my best friend and absolutely
insisted on watching my purse while we were dancing at a club. I later discovered that by "watching" she meant "leaving unattended at a table", and of course, my wallet was stolen and I had just gone to the ATM that day, had all my credit cards in there, the whole nine yards. And it was my birthday. The kicker was that she was so drunk that she forgot the whole thing and never even apologized to me. I said to mutual friends that I forgave her, but honestly, I still haven't.
Okay, this story has nothing to do with you or your puppy, but it was six years ago and I am STILL bitter. I will carry that grudge to my grave.
Moving back to you, your puppy learned to use the pad once, and that knowledge is stored in his brain somewhere. It may feel like it now, but it really isn't the same as starting from scratch. And it can be really, really hard to train yorkies, so it's possible that the timing was just a fluke and he was destined to go off the pad. Be patient and persistent and in time you WILL have a perfectly house trained dog.
