I know what you mean. I only have collars on mine while I am home which is most of the time. At any time that mine are roaming the house, I always try to put the collars on just in case if "grandma opens the sliding door and accidently let's one out" It's happened!
In fact several times

She doesn't live with us but comes for vacation once a year and she is not aware how fast a little dog can run outside. I have found my furbaby scratching the sliding door looking tired and dirty.Trying to get back in. I flipped out because I was thinking that they could have slipped through a crack in the wood fence we have.
If your pet runs away and someone takes it chances are that the person doesn't know that this is a GPS/tracking device. Or at least throws it in their house or garbage and you can find your dog asap.I don't know, I guess I'm paranoid .
Genie
Cookie
Lola
Angel