Bill Belichick may never completely shake Spygate, the scandal in which the Patriots head coach was fined $500,000 for videotaping opponent signals. Opposing players clearly haven’t gotten over it, nor have rival fanbases. Much of that is fueled by jealousy—the Patriots have been wildly successful—but there’s no denying the severity of the NFL’s punishment. In addition to Belichick’s fine, the Patriots were fined $250,000 and forfeited two draft picks.
The Spygate sanctions came down in 2007, when sports were still largely about what happened on the field, and number-crunchers hadn’t yet made the leap from their mother’s basements to the front office. Today, numbers dominate our understanding of the games, and so the logical next step would be to ask whether or not Spygate can be illuminated by crunching some stats.
As it turns out, not so much. Statistically speaking, it’s difficult to draw the conclusion that Spygate helped the Patriots win more games, according to Neil Payne of fivethirtyeight.com. It’s also difficult to completely rule it out.
New England’s performance relative to expectations in Vegas didn’t change with Spygate, according to Payne. The Patriots scored 2.4 points per game more than they “should have” before Spygate, and the same 2.4 points relative to expectations after.
“That means New England’s offensive overachievement was more likely due to great coaching and quarterback play, which persisted across both eras, than to any illicit edge.”
Payne examines wins and losses before and after Spygate, Super Bowl wins, and playoff wins. While these have all been covered over and over, Payne puts the varation in the team’s record through various t-tests. His conclusion:
“As is usually the case when dealing with real-world data, the answer isn’t totally conclusive. From a holistic viewpoint, using all repeat-opponent games in the regular season and playoffs, there hasn’t been any difference — significant or otherwise — in the Patriots’ offensive performance since the league mandated the team stop taping opposing play calls. Looking at the playoffs alone yields a more nebulous picture but also introduces methodological questions about the aptness of conventional significance testing. And we must always keep in mind that splits happen if we look for them hard enough.”
In other words, New England’s playoff record of 12-2 before Spygate and 6-6 after is troubling, but it isn’t a clear-cut sign that anything abnormal was going on.
So, basically, we’re back where we started.