I had all these thoughts and worries racing through my mind too when Scottie was neutered and then again when Casie was spayed. It's SO worrisome, even though we know everything will be okay. The days leading up to the surgery drove me nuts. Hahaha.. and my hubby had the exact same worries as yours is having. Men eh?
I took time off after their surgeries, so it's perfectly fine if you did too!! They're babies and need to be cared for like babies.
Don't stress too much (even though I know it's almost impossible not to). Just know it'll be over soon and you'll have to worry about how to keep him less active afterwards for a whole week!