I don't know if this has anything to do with it...but
The first time I used a dryer on Ginger she was 6 mos. old...I had been bathing her, and towel drying her, but her hair is thin, and it was short, so she dried really quickly.
BUT everyday, since she was about 3 months old, she's sat beside me in the bathroom while I get ready, including BLOW DRYING my hair. I think she was sooooo used to the blow dryer that it didn't even phase her when I finally turned it on her. She even seems to enjoy it now
Silly little doggie!
Anyway, my point is, perhaps just getting your furbaby used to the hair dryer for a few minutes each day, while it's not pointing in his direction, may help alot!
(I'm a big one to talk, I can't get Ginger to stop freaking out everytime I turn on the clippers...something I've just recently decided to try out on my own)
Good luck, and keep us posted! I'd love to know what finally works for you!