Rocky is the same way i feed him when i get up in the morning around 6:45am and he usually leaves some food and he will not eat when i am not there but as soon as i get home he's chomping away on his food. I notice that on the weekends he dosent eat during the day either he goes smells his food has a drink of water and he is good.
So i wouldnt worry if i were you if he were loosing weight it be a different story.