You've gotten some good advice so far: be consistent, confine them to a small area within the house when you can't give them your complete attention... also seek assistance so that you can learn to TRAIN your dogs (it makes a HUGE difference!) and exercise the heck out of them by taking them on structured WALKS... Not just running around the backyard, because that really isn't a replacement for a walk.
I only want to add one thing: I read recently that, while WE might interpret the submissive posture of a dog who has just done something we deem 'wrong' to indicate guilt (and, by extension, that the puppy understands that he was doing something bad) doesn't mean that he actually does understand. What is actually more likely is that the dog figures out that his owner gets MAD when he is CAUGHT doing that... Not that what he was DOING was bad... And so then he just tries not to be caught!
Learning that distinction really helped me to not take their behavior personally. They just haven't learned yet that they aren't supposed to be potting there!
Good luck with your babies!
