Oh dear this does not sound good to me..... at first I thought it might have been being in a new home, but you said he is underweight.
I have seen this in a friends yorkie and one day she got real sick and was taken to the ER and they found she had MVD a type of liver shunt. The only way to find out for sure is have a bile acid test (blood work).
My friends yorkie would have bad days when she would not eat much for a day and would sleep most of the day, then the next day she was active again and eating better but she was always very thin and very picky eater.
Did you get the name and # of his old vet?...if you did call them and see what he has been treated for in the pass and most recent.
I could be wrong, but I would taken him to my vet anyway being he is new to my home and will be around my other dogs. Mention the bad picky eating habit and you might also want to take a stool sample to also rule out parasites.
Keep us posted on how he is doing
