Oh goodness.
The third eyelid doing that can't be comfortable even if he isn't showing it unless everything is totally numb.
If you trust this vet, I would go with his advice and have it removed.
If you have any doubts, get a second opinion.
If it happened to Ellie and infection set in from dirt, I'd feel like it was my fault for not taking care of it early.
There is always risk with anesthesia but it shouldn't be any different than spay/neuter. Sometimes we just can't avoid it.