I live a few hours from Houston. When we go, we always visit Schlitterbahn (in Galveston) Then there's Kemah which is like a small fair. You can google Kemah and Schlitterbahn and get prices, etc. If you have children the Houston Childrens museum is excellent and it's free on Thursdays from 5-8.
Hope this helps. Also the mall in Houston is fun to check out! One place to avoid: Harwin drive. It's kinda like Chinatown? in NY but very dangerous. There have been some recent homicides. You will probably get good rates in Kemah as I wouldn't know where to recommend in Houston. I know there are good parts and bad parts to stay at.... Anyhow, good luck and congrats on your new baby!