harness for sure. i use the collars for pretty or for tags, we have one with her name in rhinestones for fancy and one with tags for going on walkies. but always always attach a leash to a d ring on a harness, harness vest or harness dress.
chattiesmom has silk step ins custom made and they are great!!
also, you can take the pup outside to pee before all shots are done, but i would recommend fencing in a small area in the backyard with chickenwire that's just for pottying purposes and make sure no other animals can enter into it.
just be careful that the pup doesn't leave your yard and go places such as walkies until all shots are done!!