I completely understand your frustration. I was a server at Ruby Tuesday for nearly two years before I finally quit. You're right; there are tons of rude people, but the nice ones at least make up for it a little bit. As a bit of encouragement, you should know that it does get easier with time. You pick up little "secrets" that will help you be faster taking the order and getting drinks and pretty much everything. Also, the better you know the menu, the easier your job will be and that also comes with time. If you decide to stay with your job, it'll probably get easier, but if you quit, you'll probably feel much less stressed. So either way, it'll work out for you.
I never regret quitting, but I think it was a good experience to have as well.
Good luck!