Dogs have strange anatomy. At the base of the penis are two glands, these are bulbous glands and they will show up as you are seeing, when the dog is excited. They are what "ties" him to the female during mating. These small marble like "balls" are NOT testicles. His testicles will be in the scrotum, back farther from these two little glands. If they haven't dropped, they will need to be surgically removed. They do have the ability to drop down into the scrotum, but then go back up again.
So those two little balls you are seeing come and go are NOT his testicles. He will have those his whole life, even after he is neutered, and they will continue to show up now and then as they are glands and are NOT removed when he is neutered.
Hope this helps clear up any misconceptions.
