You mentioned his fins look beaten up. Do you think his fins could be burnt from the ammonia buildup? How big is the tank? If it is something like this (
http://youraquariumcenter.com/library/BBT3Glamour.jpg) it's entirely too small. I'd invest in a 1/2 gallon or gallon bowl for each of them and see if he improves then.
Also I'd suggest fasting him for a 2 or 3 days. Sometimes they need to be "cleaned out."