Sounds like you and he are doing great. Just remain consistent with his training and you will both do fine.
I don't see any problem with him having a toy in with him as long as it is safe and does not have any removable eyes, nose, etc. They sell stuffed animals that have a heart beat for puppies here. I guess they are supposed to help them when they try to go to sleep.
Welcome and congratulations on your new pup.