Hi and welcome sorry about your little guy having a problem. Glad he is ok.
I started my pup right away to go outside but I have my own yard. He started before all his shots.
If you dont have your own private yard you might want to wait till he has his booster shots as they are not fully protected till the booster shot from what I have been told anyway.
If you are right with him and he has his paper to go on then you could let him out anytime.Just put him back in his pen if you cant watch him as he might end up doing a pee or poop in different place than the paper.
Hope this helps and lots of good advice will come on here. Good luck
