This is what happens when you allow your guy to run around with females all willy-nilly. Neutered or not, allowing that kind of behavior comes with consequences. Too bad you didn't figure that out
before hand.
Thank your lucky stars that him whining and following you is the worst of your problems. Allowing that kind of thing without proper supervision could be very dangerous to both the female and your little guy.