Gizmo is the same way!!! Does your baby cry/whine the whole way? Gizmo does and i feel horrible and he has gotten better over the last 6 months that we have had him.
What i started doing with Gizmo is..... Putting him in the car and then driving to a park.... Letting him walk around and play... So that he realizes that if he goes in the car he gets to go to the park or play with my family members.
I did find that little short trips a couple of times a week did help!! Try that.... OR if your baby is really nervous like u said you have to pick them up and put them in the car... A couple of times a day take Charlie out to the car and put him in it and then take him out. You dont have to start the engine... Just so he gets used to getting in and out... Then you can start turning the engine on or even going to a park.
I also give Gizmo treats when he is quiet in the car... Not all the time just once in a while so that he doesn't always expect them!
Good luck i am sorry i rambled but this is what has worked for us and i can completely relate!!!!
PM me if you want