Saturday, October 6, 2012

Preseason rankings: Do they mean anything?

Many apologies to the zero people who read this.  I've been away for a while.  Having seen a couple of postings from Bruce Pascoe, like this one, that put Arizona in the top ten before the season even started, I began to wonder whether the AP and ESPN preseason polls were worth a damn.

I remembered that Ken Pomeroy had written a post a couple of years back actually lauding the preseason poll.  I also recalled that last year, the Cats began the season ranked (#16 in both polls), only to be dropped by Week 4.   It is a fairly simple task to ask how  preseason rankings compare to actual results at the end of the season.  Obviously, it varies from team to team, but if we look at the overall pattern, how good are sports writers and coaches at predicting how good a team will be?

I suppose such rankings are created by examining last year's performance and evaluating talent retained, recruited, and lost.  Parenthetically, it's worth noting that scouts seem to have a mixed record when evaluating high school talent.  I don't know how many "five star" recruits have turned out to be much less (e.g., Josiah Turner), or how many three star guys have been really good (e.g. Kyle Fogg). It would be interesting to compare incoming player ratings to actual performance on the court, but that's for another day.

Today, I just want to look at where teams were ranked before the season began and where they ended up at the close of the season.  To this end, I compiled the AP preseason and final week rankings for the last five seasons.  I recorded whether each of the preseason top 25 teams were ranked at the end of the season and where they ranked.  Here's what I found:

The graph above shows the relationship between the ranking of teams in the preseason poll to their likelihood of remaining ranked at the end of the season.  What it shows quite nicely is that teams with a high preseason rank are very likely to be ranked at the end of the season.  For example, those teams ranked 1 through 5 in the preseason AP poll have an 88% chance of remaining ranked.  In other words, those involved in forecasting team success are highly successful at identifying the best teams out there.  However, it is much more difficult to predict teams in the lower part of the Top 25.  This trend bottoms out for teams ranked 15 to 20 in the preseason poll.  Over the last five years, only 24% of the teams ranked 15 to 20 have remained within the Top 25.

This should not be particularly surprising.  The difference between a really good team (say a 1 seed in the tourney) and one that is good but not elite (say a 10 seed) is a big difference, big enough to identify before the season begins.  In contrast, the difference between a 10 seed and a 9 seed is not particularly meaningful or foreseeable with much success.

If the Cats do get a preseason ranking in the top ten, though, this analysis would suggest that they have a good chance of remaining ranked.  For teams with ranks of 6 to 10 over the last five years, 75% of them have remained ranked at the end of the season.

No comments:

Post a Comment