I don't think you can hold them too much. Lucy wanted to be held all the time when we first brought her home...Less now that she is older but she still likes it.
I think it is great you are getting your puppy used to going places with you in your purse/carrier. That way you can take him lots of places
I introduce Lucy to anyone and everyone I feel comfortable with on our travels and as one of the other posters said, as a result she loves everyone!