Where are they going to do their business ? I would try to pick it up so they can't carry it around...I know puppies do go thru a phase of doing this...and there are things you can add to their food to make it undesirable....
My girls both tried a few bites....(found out AFTER I kissed them

) but they did grow out of it....I think the Puppy Food has the ingredients that make this such a tasty snack