Go with your heart.
I also went to a breeder only interested in females. But in my case it wasn't because that's what I was used to. It was because I had met both male and female yorkies and Was looking for a specific personality more common in female yorkies.