This was a violation of traditional coaching practice to always punt in that situation, but, after all, offenses, especially short passing offenses, are now much more reliable than when this tradition was invented decades ago. This also means the other team's offense is better, too. So, if you are Belichick, the question is, "Whose hands do I want on the ball: Tom Brady's or Peyton Manning's?"
The improvement in offenses is why you never see the "quick kick" (an unexpected punt on third down) anymore. As recently as the 1971 USC-UCLA, the Bruins surprised the Trojans by pitching out to halfback Greg Jones on third and long only to have him pull up and punt (using a sideleg topspin-inducing punting style he'd secretly practiced). With nobody on USC playing back to receive it, the ball finally rolled dead after almost 70 yards, pinning USC deep in their own territory. This helped UCLA pull out a 7-7 tie.
But there aren't many 7-7 ties anymore (at least not played on dry fields where a punt might roll a long ways), so nobody punts on third down anymore.
So, why should they punt on fourth down either?
A study by economist David Romer a few years ago argued in favor of teams going for the first down on fourth and two even on their own ten yard lines, even in the first half.
Unfortunately, according to Romer:
"Decisions to go for it on fourth down (that is, not to kick) are sufficiently rare, however, that they cannot be used to estimate the value of trying for a first down or touchdown. I therefore use the outcomes of third down plays instead."In other words, what he actually discovered that it's a good idea to go for it on third and two at your own ten yard line instead of quick-kicking it away. But we already knew that you should go for the first down on third-and-two. The odds of you making it are pretty high because the defense is playing fairly back so that you don't score a 90-yard-touchdown on them. On 4th-and-two, however, the payoff to the defense from crowding the line to prevent a short gain is much higher, so Romer's analysis isn't worth too much.
A better way to analyze fourth and two decisions is from data collected on two-point conversion. In the NFL, a team that scores a touchdown gets to either kick a one-point conversion, or, from the two yard line, attempt a two-point conversion by running or passing the ball across the goal line. The defense doesn't play back at all because there is nothing they can give up worse than a two point conversion.
At present, the odds of making it from two yards out appear to be not over 50%, because no NFL team regularly tries a two point conversion. The universal default in the NFL is the one point conversion, which has nearly a 100% success rate, putting the expected value of kicking the conversion at just a little under 1.00 points. Hence, the success rate on two-point conversions can't be much over 50 percent because NFL teams today only go for two when there is some strategic reason to do so.
However, the success rate has been going up. The NYT reported after the 2006 season:
In each of the last two seasons, N.F.L. teams have made slightly more than half of their 2-point conversions, up from less than 40 percent in the late 1990s.Eventually, an NFL offensive juggernaut might start going for two after each touchdown, but that hasn't happened yet. Coaches would rather have their players lose the game than the coach lose the game.
One thing to keep in mind, though, when apply two-point conversion rates to fourth-and-two rates is that two point conversions are usually attempted when the offense is hitting on all eight cylinders, while 4th and two attempts are made when the offense is sputtering.
So, all this theorizing is interesting, but you still have to execute on the football field, which the Patriots did not: Brady hit Kevin Faulk, running a pattern where he was coming back toward the line of scrimmage for a three yard gain, but Faulk juggled the ball and didn't grab it firmly until he was only a yard past the line of scrimmage, turning the ball over to the Colts.
Not surprisingly, Peyton Manning marched them 29 yards for the winning touchdown.