First, how much exercise does he get? I find lack of exercise to be a problem with a lot of dogs, especially the little ones which people often think don't need it, but in reality... alot of small dogs often need even
more exercise than big dogs. I know if Jackson does not get his exercise, he will get ornery and start to chew up things (and he's 2), and get into things (like toilet paper). A tired dog is a happy dog! Completely wear him out and he won't have time to think about chewing things up. I give Jackson 35-45 minute walks per day (M-F). Every night, we do trick/clicker training for at
least 15 minutes (mind games tire him out), and/or do a 10-20 minute game of fetch inside the house running down the hall. Every Saturday is agility class, dog park for 2 hours, then a trip to Petsmart/Petco/etc. A few times a week I bring him to my dads fenced in yard with their dogs where he does even more running. I realize ALL of these things aren't always capable of some people but I do believe good exercise would help 90% of peoples dogs problems most of the time. Some use treadmills in the winter time when walking is not possible. Also, training. Has he been to any obedience classes, does he know any basic commands? These are key to a happy life with your dog, IMO.
After impleting an exercise routine, start at square one again. When you ARE home, take him out every 30 minutes. I know it sounds excessive, but set him up to succeed. Then the following week, up it, and take him out every hour. Then up it again, etc. Go slowly. When you aren't home, he should be crated, so he does not have the option to chew up toilet paper. Don't set him up to fail, pick up everything and puppy-proof the house.

I'm sure he senses your frustration too (and it's understable why you are frustrated) so try to be cool and calm. He will pick up on your energy.
You say you bring him outside and he will pee and come 5 minutes later inside and poop.... well, when you know he doesn't poop when you're outside with him, why are you allowing to come into the house free roam, knowing he has to poop? I know when Jackson was a puppy, if he would not poop in the morning, he would go directly back into his crate for another 20 minutes or so, and then I'd try again. Until he pooped outside, he did NOT get that freedom.