The only thing that's really going to give you relief is to have them removed. I feel for you. I still have 2. One came in and the other is impacted. I'm not going to bother it until it bothers me
I had one surgically removed when I was in my 20's. It was sideways and impacted. I well remember being up all night for WEEKS before having it removed because I'm terrified of the dentist. After it was removed, I had dry socket. It STILL didn't hurt as bad as that tooth did.
Good luck - hope you feel better soon!