woofwoofwoof, neutering isn't a high risk surgery. They usually just put them under, make a small opening in their scrotum and remove the testicles. It doesn't take a long time and it's very routine. So you don't need to worry about your baby.
That said, I'm not an advocate for neutering just for the sake of it. But you need to decide for yourself whether your reasons are correct