I don't think it's necessarily a bad idea. The only time you'd run into a problem is when you have to transport him and the only option is using a crate. However, the litter box/crate idea is probably a good solution to that.
Have you tried an ex-pen? We crated Franklin for the first couple of nights but we decided he needed more room to roam around. Ex-pens are perfect and he almost NEVER has accidents in his ex-pen. With pee anyway, we're still working on the poop.