I agree that he probably sleeps most of the time you're gone--mine do that and then have lots of energy to expend in the early evening.
When we're going to be gone for a few hours on weekends (I'm usually with them during the week), we might leave the tv on a "safe" channel. If it's Animal Planet, we leave the screen on because our cat and one of the yorkies like to watch the animals. Sometimes we may leave just the sound on or a radio at a low volume, making sure that alarming noises won't be included in the programming. (Piper freaks out if suspenseful music is used in the background of a tv show or movie.) A lot of tv's can be programmed to only stay on for a given period of time, which might be useful.
My mom has yorkies, as well, and also owns a cockatiel. The dogs don't physically interact with the bird, but they all like watching each other and compete to be the best security guards and noisy commentators on household activities.
As mentioned, toys and sometimes a strategic treat dispenser can help.
I really can't think of much else since my experience has been the potato chip syndrome: can't have just one yorkie.
Good luck!