Letting him walk on the edge of the yard and sniff mailboxes is perfectly acceptable, IMO. As long as you clean up his poop, it should be ok for him to poop there. After all, it's really tiny poop!
I take bags with me everywhere and never leave my pup's poop anywhere, even in my own yard.