Milo was like that. Family told me that even when he was home with them he spent the whole time up in the window looking out at the driveway.
Then we brought home Remy. DH said yesterday when I went to the bank that Remy waited at the door for me for a little while but Milo was happy and not worried at all.
Milo use to shake and pant in the car too. Every time. Now he is all calm and content because Remy is with him. Remy loves the car.
Good luck, I know how it tugs at your heart to feel that you are causing all this distress.