Hi. I'm new here. We are planning to get a Yorkie puppy in a few months, and I've been doing research on the breed as well as breeders. We just lost our Maltese a couple of weeks ago. He was only 5 years old but had numerous, on-going health problems and had to be put to sleep. After doing my research, I believe we bought our Maltese from a puppy mill.
It does appear that you have done the same. I can understand your wanting to return the puppy and get your money back. I can also understand your desire to give your puppy a good life, probably a much better life than what the puppy mill breeder can offer.
My personal opinion is that you should keep your puppy. Although we didn't have our puppy mill dog as long as we had hoped, I know we gave him a good life filled with lots of love and kindness. You can do the same for your dog. It's obvious that you love your dog and will take care of him. Who knows what will happen to the puppy if you give him back.
Enjoy him.