Friday, March 18, 2011

Bracketology-ology: a Report Card

Selection Monday revealed that the selection committee this year was a little bit more closely aligned with RPI than in years past. Unlike last year's at-large picks of UA Little Rock and UW Green Bay (two teams that received automatic bids this year), there weren't any teams selected that one could really argue were long shots. The team with the lowest RPI selected for an at-large bid was Purdue (49).

So how did the S-Factor stack up this year to the experts?

The S-Factor missed only three teams this year (Dayton, Middle Tennessee, Vanderbilt), which isn't too bad. But of course total number of teams in the tournament is only part of the mission of the S-Factor.

The S-Factor was better at predicting seeds than any other predictor, human or computer. Charlie Creme, ESPN's legendary bracketologist, correctly predicted the seeds of 31 teams, but the S-Factor did him one better with 32.

Other metrics of comparison are shown in the tables below. There were 21 predictions that I could find this year. I assigned a point total to each prediction corresponding to its ranking for each category. The point system works similar to the AP poll: the first place prediction gets 21 points, the second gets 20, and so on. These points were then tallied to determine an overall bracketology champion, shown at the end of this post.

The 21 bracket predictions are as follows:

1. Rating Percentage Index. Compiled by Jerry Palm.
2. Jeff Sagarin ratings; a synthesis of the ELO CHESS and PURE POINTS models. Compiled by Jim Sukup.
3. Sagarin ELO CHESS (see above)
4. Sagarin PURE POINTS PREDICTOR (see above)
5. Kenneth Massey's ratings
6. Stephen Jens
7. Sonny Moore's computer power rankings
8. Cody Kirkpatrick's ratings using David Wilson's method (
9. Sportheory
10. - RPI rankings (which differ from Palm's)
11. - power rankings
12. pilight's points only - pilight's field of 64 model without the exceptions, such as for teams that are sub .500 in conference
13. Warren Nolan's power index
14. The S-Factor

Actual bracket predictions:

15. Charlie Creme
16. Jerry Palm
17. pilight's field of 64
18. RealTimeRPI's bracket
19. my bracket prediction

Polls - including the additional votes

20. AP poll
21. ESPN Coaches poll

Metric 1: predicted teams that actually made the tournament:

1RealTime RPI (RPI)6398.4%21
2College Women's Hoops official bracket6296.9%20
2Jerry Palm6296.9%20
2RPI (Palm)6296.9%20
5Charlie Crème6195.3%17
5RealTime RPI bracket6195.3%17
5College Women's Hoops S-Factor6195.3%17
5Ken Massey6195.3%17
5pilight points only6195.3%17
10pilight's field of 646093.8%12
10Sagarin ELO Chess6093.8%12
10Cody Kirkpatrick (
14Stephen Jens5992.2%8
14Sonny Moore5992.2%8
17RealTime RPI (Power)5890.6%5
17Sagarin Pure Points 5890.6%5
19Warren Nolan5585.9%3
naAP pollnana0
naESPN/Coaches pollnana0

Metric 2: predicted teams correctly seeded (entire field)

1College Women's Hoops S-Factor3250.0%21
2Charlie Crème3148.4%20
3College Women's Hoops official bracket2945.3%19
5Jerry Palm2843.8%17
5Ken Massey2843.8%17
5Sagarin ELO Chess2843.8%17
5Stephen Jens2843.8%17
9Cody Kirkpatrick (
10RealTime RPI (RPI)2539.1%12
10RPI (Palm)2539.1%12
12pilight points only2437.5%10
14Sonny Moore2132.8%8
15Sagarin Pure Points 1929.7%7
16pilight's field of 641828.1%6
17ESPN/Coaches poll1523.4%5
17RealTime RPI bracket1523.4%5
17RealTime RPI (Power)1523.4%5
20AP poll1218.8%2
20Warren Nolan1218.8%2

Metric 3: predicted teams seeded within one seed of actual result (entire field)

1College Women's Hoops official bracket5687.5%21
2Charlie Crème5484.4%20
3Cody Kirkpatrick (
4College Women's Hoops S-Factor5179.7%18
4Sagarin ELO Chess5179.7%18
6Jerry Palm5078.1%16
7Ken Massey4671.9%15
7RPI (Palm)4671.9%15
9RealTime RPI (RPI)4570.3%13
9RealTime RPI bracket4570.3%13
12Sagarin Pure Points 4367.2%10
14pilight points only4164.1%8
14pilight's field of 644164.1%8
16Stephen Jens3960.9%6
17Sonny Moore3656.3%5
18RealTime RPI (Power)3148.4%4
19AP poll2945.3%3
20Warren Nolan2742.2%2
21ESPN/Coaches poll2234.4%1

Metric 4: predicted teams correctly seeded (top 12 seeds)

1Charlie Crème2654.2%21
2College Women's Hoops S-Factor2347.9%20
2College Women's Hoops official bracket2347.9%20
4Ken Massey2245.8%18
6Jerry Palm2143.8%16
6Sagarin ELO Chess2143.8%16
6Stephen Jens2143.8%16
6Cody Kirkpatrick (
10RPI (Palm)1939.6%12
11RealTime RPI (RPI)1837.5%11
11pilight points only1837.5%11
14ESPN/Coaches poll1531.3%8
15Sonny Moore1429.2%7
16Sagarin Pure Points 1327.1%6
17pilight's field of 641225.0%5
17AP poll1225.0%5
19RealTime RPI bracket1122.9%3
19RealTime RPI (Power)1122.9%3
21Warren Nolan816.7%1

Metric 5: Predicted teams within one seed of actual result (top 12 seeds)

1College Women's Hoops official bracket4287.5%21
2Charlie Crème4083.3%20
2Cody Kirkpatrick (
4College Women's Hoops S-Factor3879.2%18
4Sagarin ELO Chess3879.2%18
6Jerry Palm3675.0%16
7Ken Massey3368.8%15
7RPI (Palm)3368.8%15
7RealTime RPI (RPI)3368.8%15
7RealTime RPI bracket3368.8%15
13pilight points only3164.6%9
13pilight's field of 643164.6%9
15Sagarin Pure Points 3062.5%7
16AP poll2960.4%6
17Stephen Jens2858.3%5
18Sonny Moore2756.3%4
19ESPN/Coaches poll2245.8%3
20RealTime RPI (Power)1939.6%2
21Warren Nolan1837.5%1

Metric 6: predicted teams correctly seeded (top 6 seeds)

1College Women's Hoops official bracket1770.8%21
1Charlie Crème1770.8%21
3Ken Massey1666.7%19
3Stephen Jens1666.7%19
5RPI (Palm)1562.5%17
7Sagarin ELO Chess1458.3%15
7Jerry Palm1458.3%15
9ESPN/Coaches poll1354.2%13
10College Women's Hoops S-Factor1250.0%12
10Sonny Moore1250.0%12
12RealTime RPI (RPI)1145.8%10
12AP poll1145.8%10
15Cody Kirkpatrick (
15pilight points only1041.7%7
15pilight's field of 641041.7%7
15RealTime RPI (Power)1041.7%7
19RealTime RPI bracket833.3%3
19Sagarin Pure Points 833.3%3
21Warren Nolan729.2%1

Metric 7: predicted teams seeded within one seed of actual result (top 6 seeds)

1College Women's Hoops official bracket24100.0%21
1Charlie Crème24100.0%21
3Sagarin ELO Chess2395.8%19
3Jerry Palm2395.8%19
3AP poll2395.8%19
6College Women's Hoops S-Factor2291.7%16
6RealTime RPI (RPI)2291.7%16
6Cody Kirkpatrick (
9RPI (Palm)2187.5%13
11pilight points only2083.3%11
11pilight's field of 642083.3%11
11RealTime RPI bracket2083.3%11
14Ken Massey1979.2%8
16Stephen Jens1875.0%6
16ESPN/Coaches poll1875.0%6
18RealTime RPI (Power)1770.8%4
18Sagarin Pure Points 1770.8%4
20Sonny Moore1666.7%2
21Warren Nolan1458.3%1


The overall winner? My bracket guess! Bested Charlie Creme by 3 points.

And once again, the College Women's Hoops S-Factor beats all other computer predictions.

Bracketology-ology Final Summary

rankmetric no.--->1234567sum of pts
1College Women's Hoops official bracket20192120212121143
2Charlie Crème17202021202121140
3College Women's Hoops S-Factor17211820181216122
4Jerry Palm20171616161519119
5Sagarin ELO Chess12171816181519115
6Ken Massey1717151815198109
7RPI (Palm)20121512151713104
8Cody Kirkpatrick (
10RealTime RPI (RPI)2112131115101698
11Stephen Jens817616519677
12pilight points only1710811971173
14RealTime RPI bracket1751331531167
15pilight's field of 6412685971158
16Sonny Moore8857412246
17AP poll02356101945
18Sagarin Pure Points 5710673442
19ESPN/Coaches poll0518313636
20RealTime RPI (Power)554327430
21Warren Nolan322111111

No comments:

Post a Comment