Mine do this some times, so I know how you and your pup feels.
I've never done it but I've heard that feeding can pumpkin helps. Also it very important that you keep his bottom trimmed. Cut all of the long hair away from his pooper, that will help when you goes it wont get bound up. Below is a site that may help.
What Helps Soften My Dog's Stool? | eHow