I know the feeling. I work full time and do not have a husband or a roommate to help care for Scooter. The first time you leave them is so hard. Scooter is ten months old now and does great. I work about 15 minutes from my house so I come home and spend lunch with him.
As long as someone is able to let your baby out on some sort of regular schedule, they adjust great. I don't use a crate with Scooter, he originally had a bathroom (with a puppy gate up instead of a closed door) and now has moved to using an xpen to keep him in my kitchen. I only leave him for about 4-5 hours at a time and he does quite well with it now. The beauty of giving him a room instead of a crate is that I had room for a puppy pad, his bed, a water bowl, and some toys. We have mastered potty training and he no longer needs the puppy pads.
One thing you may want to consider is Doggie Daycare. I didn't want Scooter getting bored at home and there are days that I have lunch meetings or can't make it home to check on him. I now take him once or twice a week. The daycare is close to my office and they have extended hours. They take such good care of him and he is with dogs of similar size and temperment. He enjoys it so much that he starts barking at traffic if they aren't moving fast enough. He comes home so tired from a full day of play that he tends to fall asleep in my lap on the drive home.
Daycare could be a great option so that you and your neighbor can have a little time to do other things. I think it is great that you have a neighbor willing to come over daily that you trust enough to get in your house and take care of your baby. You may also want to consider leaving puppy pads out for your baby as you continue potty training. Good luck.