Verducci Effect a Myth? Q&A with Brian Burke of Advanced NFL Stats

We welcome Brian Burke, who has been so gracious to take time out of his busy schedule to answer a few questions for us. Before dedicating time to his hobby of football at Advanced NFL Stats, Brian flew 21 combat missions in the Navy. Brian tackles a lot of game theory, and his site is a must read for football fans.

RotoSavants: What observations did you make when you wrote about the "Verducci Effect"?

Brian Burke, Advanced NFL Stats: My hunch was that these injury-prediction theories were all biased by a number of factors. Injuries are by their nature very random, so I am skeptical whenever someone claims to have a valid prediction system. The bottom line is that injury-prediction systems usually fail to account for systematic biases in their analysis.

First, it's always true that athletes will tend to get more prone to injury as they age. Therefore, any comparison of a year "after" something else, such as RB carries or additional innings pitched, will appear to have a slightly higher injury rate than other years.

Second, high-IP or high-carry years are by definition healthy, non-injury years. Whenever you make observations about seasons "after" those years, you are also making observations about "other than" those healthy years. You're removing known healthy years from the data sample, and years left over in the sample will appear to have higher than normal injury rates. But the higher rates are just an illusion caused by the exclusion of healthy years.

It all depends on the type of analysis performed. It's not perfectly clear what the exact methodology the Verducci effect is based on, but I think it's fair to expect explicit and rigorous research on it before we buy into it.

RS: You talk a lot about different types of bias in your articles, including the popular 'Myth of 370', can you explain why that's important for a fantasy player?

Burke: The Curse of 370 is a classic example of what's known as "multiple endpoints." Readers should always be skeptical whenever someone makes a claim based on a very exact cut-off of carries, innings, or whatever. What's going on is that the analyst is intentionally choosing a cutoff that makes his claim appear true, but only due to a statistical quirk or fluke that occurred in the past and will not hold for future events. For example, if I said Jamal Lewis has had more games of over 70 yds rushing than any other RB in history, you'd think he's bound for the Hall of Fame. But Lewis might only be 17th in games with over 75 yds because he happened to have lots of mediocre games.

For fantasy players, the debunking of myths like 370 mean that all top players can be expected to regress to the mean following high-performing seasons. The higher the performance, the stronger the regression. So the best bet is to go after the top guys and not worry about curses, injuries, etc. Sure, they'll tend to regress, but so do all the other guys too.

RS: One of the reasons we can't dig deeper into some methodologies is because they are 'proprietary'; do you get better observations out of academic papers?

Burke: Actually, I don't have blind faith in academic papers either. Many of them are peer-reviewed by professors who don't understand the nature of the sport, and gross errors in logic get through. I'm very, very skeptical of anything that calls itself proprietary. Whenever I read that word, I think "unverifiable and off-limits to criticism." I can understand why people want to protect their intellectual property, but they should also understand why readers should be skeptical.

RS: I've heard the 'age 27 peak', expect regression from postseason pitchers as facts across many platforms. What would be the best advice to understand the difference between causality and correlation?

Burke: I would agree there is probably a peak physical age, and players will tend to decline past their late 20s, especially those who are primarily dependent on their speed instead of strength or power.

Correlation does not equal causation. This is the first rule of inferential statistics, and it is violated 100 times a day in sports, business, and politics. All players tend to regress toward their mean--their true performance level from which their stats will vary from year to year. Phenomenal (or "outlier") pitching seasons are characterized by lots of things going right all at once--health, luck (BAPIP), solid defense, weak opponents, etc. Those things are far less likely to continue than they are to revert to normal, so we should expect pitchers with the very best seasons to regress the most the following year. And since we find the best pitchers leading their teams in the postseason, very often they will appear to have bigger drop-offs in performance the following year. The tendency to decline is not due to the postseason at all, just due to the fact they had a higher perch from which to regress. Playoff appearances and statistical declines may be correlated, but one doesn't necessarily cause the other.

RS: Finally I loved the idea and inspiration behind your Fantasy Football Projection system. This is one must-have projection system, even just for name only purposes...

Burke: "Koko" the fantasy football monkey is named after George Costanza's nickname at Kruger Industrial Smoothing. The system is a simple regression from the previous year's fantasy performance and is intended to be the bare minimum baseline of predictive accuracy. If other fantasy predictions aren't much better than Koko, they are a waste of time. My hunch is that outcomes in fantasy football are so subject to randomness, such as freak injuries, that even a monkey can be just as competitive as any expert out there.

When not claiming Fantasy expertise, Ryan writes about Mid-Major College Basketball at SienaSaintsBlog.com.