I wouldn't wait either. Maddie was doing this and I took her to the vet. The first thing he said when I descibed her isolating behavior was, "She is pain, let's find out why." Come to find out she had a severe inner ear infection, she was showing no other signs except her behavior.
I think you'd feel much better if you try to take your little guy to the vet asap.
