2013 Bracketology-ology

How did everyone do?  Depends on methodology, as always.

I analyzed 7 bracket predictions and 21 ranking models (listed in this post).  In 2011 I used seven metrics to judge which predictions were the best, and I have chosen these metrics once again this year:

•  Number of teams that were correctly selected
•  Of the entire field, number of teams that were correctly seeded
•  Of the entire field, number of teams seeded within one seed of the actual seed
•  Of the top 12 seeds (roughly the at-large cutoff), number of teams correctly seeded
•  Of the top 12 seeds, number of teams seeded within one
•  Of the top 6 seeds (roughly the AP and Coaches poll predictive cutoff), number of teams correctly seeded
•  Of the top 6 seeds, number of teams seeded within one

Here are the lists.

1. Correctly selected



2. Correctly seeded (entire field)



3. Seeded within 1 (entire field)



4. Correctly seeded (top 12 seeds)



5. Seeded within 1 (top 12 seeds)



6. Correctly seeded (top 6 seeds)



7. Seeded within 1 (top 6 seeds)



In 2011 I used a Borda count method (assigning a point value to a ranking and summing the points, like how the AP poll does it) to combine rankings for all seven metrics.  In 2011 the S-Factor was third best at prediction.  The S-Factor was a bit worse this year, coming in fifth, tied with pilight's Field of 64 method.   Still, by the Borda count method, the S-Factor remains the best numeric-only method at tournament prediction.  The bracket produced by RealtimeRPI.com takes top marks by this method.




This year I calculated a ranking based on Paymon points, which is how the Bracket Project analyzes the men's NCAA tournament bracketologists.  Paymon's method gives three points to each team selected, two points to each team correctly seeded, and one point to each team seeded within one seed of the predicted seed.  By the Paymon method, S-Factor comes in fourth among all methods, and is still the best numeric-only method.  Charlie Creme's bracket powers to victory by this method, as no other method or bracket came close to the number of correctly seeded teams.




A third way of comparing tournament predictions is to simply add up the number of correct picks falling into the above seven metrics; unlike in the Paymon method, this method captures the complexity and relative importance of teams seeded 1-6 and 7-12 over the teams seeded 13-16.  Another way of expressing this method, in formula format:

Where

• x_i  is 2 if predicted team i’s seed matches actual team i’s seed, 1 if predicted team i is seeded within 1, and 0 if predicted team i is two or more seeds off;
• y_i  is 3 if actual team i’s seed is 1 through 6, 2 if actual team i’s seed is 7 through 12, and 1 if actual team i’s seed is 13 through 16;
• z_i is 1 if actual team i was predicted in the tournament, 0 otherwise.

Charlie Creme's bracket is once again the best by this method.  S-Factor comes in sixth, ahead of all other numeric-only methods.  



  


See the data used for this post here! If you catch an error, let me know.