I don't think you're insane, because I'm insane too!!
My little man is getting older and he wears a sweatshirt or T shirt in the house during the day and his jammies at night. Of course he has sweaters and coats for outside too. My son laughs and I told him he's just jealous because he can't wear little blue fuzzy jammies anymore!