Who Got Screwed?

I analyzed five bracket predictions and 24 team ranking systems for my forthcoming look at bracketology-ology ("metabracketology" is maybe a better name, I think I'll go with that).  They are as follows:

Brackets:
1. Charlie Creme's bracketology
2. My bracket
3. RealtimeRPI's 65-team bracket
4. Omni Rankings bracket
5. College Sports Madness bracket

Ranking systems:
6. S-Factor
7. WBBState's "The State"
8. RealtimeRPI power rankings
9. Sagarin ratings
10. Sagarin ELO CHESS only
11. Sagarin PURE POINTS only
12. Sagarin GOLDEN MEAN only
13. Massey ratings
14. Massey's power ratings
15. Omni Ratings
16. Charlie Burrus's College Basketball Rankings (Women)
17. Warren Nolan's "NPI"
18. SporTheory's rankings
19. Sonny Moore's Computer Power Ratings
20. CJBratings.com composite ratings
21. CJBratings.com "Win ratings"
22. Stats.com "UPS Team Performance Index"
23. pilight's "field of 64" (rebkell boards)
24. RPI calculated by NCAA
25. RPI calculated by Warren Nolan
26. RPI calculated by RealtimeRPI
27. Win-loss percentage (from NCAA)
28. AP poll
29. Coaches poll

For the ranking systems, I created a list of 64 teams that would have been predicted as being in the tournament by that system; that is, the top 32 at-large teams along with the 32 automatic bids.  I averaged the seed predicted by each bracket or ranking system, and then took the difference between the average seed and the actual seed received by each team.






By this method, BYU lays claim to the most disrespected team in the country.  The average seed predicted was between 8 and 9, but the Cougars received a 12-seed from the committee.  James Madison, Albany, Gonzaga, and Oklahoma round out the top five most disrespected teams by average seed, while Penn State, Vanderbilt, Michigan State, Purdue and Georgia were the five teams most favored by the committee. 

"But to average 8.38, some insane method had to have been forecasting a 7 seed or higher for BYU?" you might ask.  Well, not many besides this one weirdo forecasting method had BYU that high. The high average is partially due to the outlier methods like Warren Nolan's NPI, WBBState's The State, Stats TPI, and NCAA W-L percentage, methods that pay more attention to the team's record than the schedule difficulty.  Removing those methods from the average yields the following list:




This list still shows BYU and James Madison on top, but it knocks Albany's average seed down to 13.60, good for only seventh most disrespected team.  By this method, Vanderbilt is the most privileged team, seeded two seeds above their average.

Oklahoma gets included on both of these lists as a disrespected team, but many would say they're lucky to even be in the tournament this year.  The tables above measure distances from average predictions, but they do not take into account predictions that do not include teams that made the tournament.  So we also need to see how many predictions forecasted a berth for the at-large teams.




This table ranks the teams by the number of selection methods that predicted their selection. By itself, this list of model consensus could have been a useful forecasting tool for predicting the bubble.  The consensus only missed on two teams this year, Florida and either Vanderbilt, St. John's or St. Joseph's, which all tied at 17 predictions apiece.  20 methods predicted Rutgers would make it, and 25 predicted Bowling Green would make it, which lends some truth to Bowling Green's claim as the most egregious selection committee error.

All in all, it was a very reasonable field of 64 teams that were selected by the NCAA this year, in contrast to the Kansas debacle of last year, where Kansas's inclusion into the tournament was predicted by no one.  Charlie Creme of course got all 64 teams right, which is a feat that should be applauded.

More metabracketology to come soon.