When my friend would come over without her female he would go bonkers. He would still smell Mimi on her owner Carmen and get antsy. So I would definitely have it done.
By the way there are places where you can get the spaying/neutering for very low cost. Some places like Bide a Wee I think will go on a sliding scale in regards to your income so check them out.
Here is a place that you can get the certificate from so you can get the spay/neuter for cheaper than the norm:
http://www.muffins.org/