Monday, November 16, 2009

Is Bill Belichick Great?

Unfortunately, I missed the latest Pats game. Well, to be honest, I've missed every Pats game... but that's neither here nor there (if we're being specific, it's there and not here...) However I do hear that Bill Belichick made a questionable call on 4th and 2 in his own territory. Pop economist Steven Levitt loved the call, saying that it was statistically the correct thing to do.

But was it? The claim is based on a paper by respected economist David Romer, which I haven't read, but assume deals with correlations between punting versus going for it and points scored. That is, non-experimental data: I don't think that anyone is actually randomly assigning 4th downs to punt on and 4th downs to go for it (following Romer's rules for what yardages are appropriate). This means there's no accounting for a coaches intuition, his observations about the relative levels of fatigue or any other hunch that can't be captured by statistics.

Glancing at the paper, Romer also used data only from the 1st quarter and used the expected yardage gain from 3rd downs as the expected gain from 4th downs (because there are too few actual 'go for it' 4th downs for analysis). So even if the paper was correct about punting to be a bad option, punting might only be a bad option in the 1st quarter and if the opposing defense plays the same way it would as if it were 3rd down (unlikely).

The data, then, might not be so clearly on Belichick's side after all. I wouldn't be surprised at all if the Pats had a resident statistician who extended Romer's analysis to 4th quarters and 4th downs, but there is still no getting around the fact that coaches' unobservable impressions can't be accounted for in the data but could still be important.

So what were your impressions? Did the Pats look like they were going to succeed or did it look like a bad idea from the minute they walked on the field?

1 comment:

  1. No, no, no, I yelled at the TV as the Pats surprisingly lined up but not in punt formation. But the Patriots really lost the game much earlier, when Brady lost an ill-considered pass to Moss in the endzone--that was, of course intercepted; and when Maroney fumbled in the endzone, recovered by the Colts. If the Pats had gotten a field goal either of those times, they would have won the game.