My yorkie HATED the leash.
He would refuse to go to the bathroom while on one. He made you miserable if I would chain him out or even walk him. He'd pull so hard he'd start making this strange choking sound. I tried all kinds of things,but I ended up letting him go potty loose in my yard (with supervision of course). Honestly, even two months ago, I had him on a leash on vacation and he held his waste for almost a day! My point, not to be a downer, but I wish I never had given in. He took off and was killed almost two weeks ago.
So no matter what you do, DO NOT LET HIM LOOSE! No matter how safe you are. A little tough love and more work may have made all the difference in my case. I just dont want to see anyone make the same mistake.
I wish you the absolute best on teaching your little guy. My new one is being a pain, but I wont give in this time.