Maybe you can take him with you on visits to your breeder to get to know her on different territory. And also, get him used to her smell on you.
When you first bring her home, have them meet on neutral territory, not in your home.
It can take some time, but I have noticed in my furfamily, that the boys have accepted new fur members much more quickly than the girls do