my friend just moved from La Jolla to Santa Monica. the weather out there is *awesome*. in may, don't expect it to be too hot, it'll be warm though. even though you said you weren't a "partier", you should check out the gaslamp district anyway. there are TONS of lounges, clubs, upscale bars, shopping, restaurants, etc.
have fun!
edit:
http://www.gaslamp.org/