Oh boy, I know how you feel. Its so frustrating! It happened to me and I thought I would have a heart attack.
I usually try to lock all the doors when I am at someone's house but I know sometimes its hard to watch every single one because people tend to come in and out from different doors all the time. So now if I feel my baby is not safe in the house and I dont trust people around, I use his gate.