Yes, size can matter. If you're showing or breeding, size does matter.
As a pet, size can matter too. If you have a larger yorkie and want a second one - but you travel - often times people might look for a smaller one as the second yorkie. If a person is "older" (like my mother in law), she wants to make sure that the yorkie is big enough to jump up on furniture, but small enough so that it will be comfortable for her to pick up (so, ideally, in the 4-5 pound range).
So, yes, size can matter - but it certainly doesn't deter how big the love is, bc that always trumps size!