Thank you everyone. I think I'm going to start letting him roam a little at a time. I don't have stairs to worry about, but every room is carpeted except the kitchen. I'll just have to try it.
I tried it a few months ago, but he would go to my little girls room and pee/poop in there. I also just recently got him neutered, so that should help with him wanting to mark things. But, it's been a while since then and he doesn't have accidents anymore. And yes, I guess the worse thing that would happen would be him pottying somewhere.
And if that happens, I'll try again when he's a little older.