I agree. You need to tell your dad that you love him but his comments hurt your feelings. That you love your husband and have no intention of leaving him. Spend time with your dad away from your husband when he is in town. If he continues with the behavior, well then, you'll have a choice to make. I'm sorry.
Big hugs.