When Lucas was 12 weeks, he was 3.5 or 4lbs. He is now 1 year and 4 months, and holding steady at 6 lbs. So if your boy is only 5 lbs at 12 weeks, I don't think he'll be anywhere near 20lbs. Lucas was off the chart too, I believe, but he is now totally average sized
