The natural disaster itself is not anyone's fault.
The problem is, the levees that surround New Orleans. That is what REALLY caused damage. These levees were lacking any sort of "real" engineering that could prevent water from getting through.
Basically, think of those levees as a wood/stick damn built by beavers. In fact, here is a picture of actual New Orleans levees taken in 1995.
http://www.gpc.edu/~pgore/myphotos/neworlen/levee2.gif http://gpc.edu/~pgore/myphotos/neworlen/levee1.gif
So this problem was ignored for years and years. Then Katrina came.
Someone or some people ignored the issue. So there is going to be some finger pointing.
Another problem, was the lack of immediate response to the millions of people in need. Those are the issues that Bush is taking responsibility for. Not the natural disaster itself, but the aftermath and the pre-existing problems that lead New Orleans to become flooded. Stuff that was preventable.