Dexter was like this.
He was trained to use pee pads, so when it was time to teach him to do it outside he looked guilty and wouldn't go if anyone was watching. If you think about it, it's all he's ever known to pee inside, and if he has done it anywhere else other than the pee pad then he gets scolded.
I waited for him to start going and then looked at him, so he had no chice but to finish then whilst I was looking, he's totally fine now!
He also didn't like wet grass, rain, wind, puddles etc. I think some of these tiny dogs are just fussy
Also, I was told by someone not to praise him whilst he was doing his business so as not to distract him, because I was doing the exact same thing you are. Let him get on with it, then praise him.
I hope that helps. I thought I could actually pull out the hair from my head when toilet training Dexter! But it gets better, he's great now and so will Jaeger be
