Wow, that is a big strain on you, financially and emotionally! I can't imagine it.
I know you're feeling guilty about it, but if you can't love him the way that he needs to be loved, then you should make it right by him. You know? He's got to be happy, with your three, being there any playing with him. But, you have to do what's right for the both of you.
You've had him for three months and you're still not in love with him, then maybe it isn't a good match?
I wish you the best of luck! I hope you can find what's right with you and him soon!