Only you know what is best for Baxter. I think it's awesome that you are putting his needs first.
When I was 16yrs old, I got a dog and was forced to move out on my own because of it. I found myself in the same situation as you are in. A very busy girl, trying to make ends meet and finish school. I felt bad that I didn't have much time for my puppy, but I did the best that I could. That dog ended up being THE best dog ever. She was my bestfriend, and everyone loved her. She was SO intelligent. I had her up until the summer of 2007, 15 yrs. Life eventually settled down for me, I finished school, I got married and had 3 kids...I quit work to become a stay at home Mom and I was able to spend tons of time with her after those first 3 yrs that I had her.
I guess what I'm trying to say is...all that you can do is your best. If you feel that you can really put the effort forth to keep him happy now, then you should keep him. He is going to be around for many, many yrs. Your life will change. It won't always be so hectic. But if you honestly feel that he would be better elsewhere, then let him go. Maybe you can even find someone that would allow you to visit him from time to time.
Good Luck with your decision.