Bracketology-ology: a Report Card

Selection Monday revealed that the selection committee this year was a little bit more closely aligned with RPI than in years past. Unlike last year's at-large picks of UA Little Rock and UW Green Bay (two teams that received automatic bids this year), there weren't any teams selected that one could really argue were long shots. The team with the lowest RPI selected for an at-large bid was Purdue (49).

So how did the S-Factor stack up this year to the experts?

The S-Factor missed only three teams this year (Dayton, Middle Tennessee, Vanderbilt), which isn't too bad. But of course total number of teams in the tournament is only part of the mission of the S-Factor.

The S-Factor was better at predicting seeds than any other predictor, human or computer. Charlie Creme, ESPN's legendary bracketologist, correctly predicted the seeds of 31 teams, but the S-Factor did him one better with 32.

Other metrics of comparison are shown in the tables below. There were 21 predictions that I could find this year. I assigned a point total to each prediction corresponding to its ranking for each category. The point system works similar to the AP poll: the first place prediction gets 21 points, the second gets 20, and so on. These points were then tallied to determine an overall bracketology champion, shown at the end of this post.

The 21 bracket predictions are as follows:

1. Rating Percentage Index. Compiled by Jerry Palm.
2. Jeff Sagarin ratings; a synthesis of the ELO CHESS and PURE POINTS models. Compiled by Jim Sukup.
3. Sagarin ELO CHESS (see above)
4. Sagarin PURE POINTS PREDICTOR (see above)
5. Kenneth Massey's ratings
6. Stephen Jens
7. Sonny Moore's computer power rankings
8. Cody Kirkpatrick's ratings using David Wilson's method (talismanred.com)
9. Sportheory
10. RealTimeRPI.com - RPI rankings (which differ from Palm's)
11. RealTimeRPI.com - power rankings
12. pilight's points only - pilight's field of 64 model without the exceptions, such as for teams that are sub .500 in conference
13. Warren Nolan's power index
14. The S-Factor

Actual bracket predictions:

15. Charlie Creme
16. Jerry Palm
17. pilight's field of 64
18. RealTimeRPI's bracket
19. my bracket prediction

Polls - including the additional votes

20. AP poll
21. ESPN Coaches poll



Metric 1: predicted teams that actually made the tournament:

Rank		No.	Pct.	pts.
1	RealTime RPI (RPI)	63	98.4%	21
2	College Women's Hoops official bracket	62	96.9%	20
2	Jerry Palm	62	96.9%	20
2	RPI (Palm)	62	96.9%	20
5	Charlie Crème	61	95.3%	17
5	RealTime RPI bracket	61	95.3%	17
5	College Women's Hoops S-Factor	61	95.3%	17
5	Ken Massey	61	95.3%	17
5	pilight points only	61	95.3%	17
10	pilight's field of 64	60	93.8%	12
10	Sagarin	60	93.8%	12
10	Sagarin ELO Chess	60	93.8%	12
10	Cody Kirkpatrick (talismanred.com)	60	93.8%	12
14	Stephen Jens	59	92.2%	8
14	Sonny Moore	59	92.2%	8
14	Sportheory	59	92.2%	8
17	RealTime RPI (Power)	58	90.6%	5
17	Sagarin Pure Points	58	90.6%	5
19	Warren Nolan	55	85.9%	3
na	AP poll	na	na	0
na	ESPN/Coaches poll	na	na	0

Metric 2: predicted teams correctly seeded (entire field)

Rank		No.	Pct.	pts.
1	College Women's Hoops S-Factor	32	50.0%	21
2	Charlie Crème	31	48.4%	20
3	College Women's Hoops official bracket	29	45.3%	19
3	Sagarin	29	45.3%	19
5	Jerry Palm	28	43.8%	17
5	Ken Massey	28	43.8%	17
5	Sagarin ELO Chess	28	43.8%	17
5	Stephen Jens	28	43.8%	17
9	Cody Kirkpatrick (talismanred.com)	27	42.2%	13
10	RealTime RPI (RPI)	25	39.1%	12
10	RPI (Palm)	25	39.1%	12
12	pilight points only	24	37.5%	10
13	Sportheory	23	35.9%	9
14	Sonny Moore	21	32.8%	8
15	Sagarin Pure Points	19	29.7%	7
16	pilight's field of 64	18	28.1%	6
17	ESPN/Coaches poll	15	23.4%	5
17	RealTime RPI bracket	15	23.4%	5
17	RealTime RPI (Power)	15	23.4%	5
20	AP poll	12	18.8%	2
20	Warren Nolan	12	18.8%	2

Metric 3: predicted teams seeded within one seed of actual result (entire field)

Rank		No.	Pct.	pts.
1	College Women's Hoops official bracket	56	87.5%	21
2	Charlie Crème	54	84.4%	20
3	Cody Kirkpatrick (talismanred.com)	53	82.8%	19
4	College Women's Hoops S-Factor	51	79.7%	18
4	Sagarin ELO Chess	51	79.7%	18
6	Jerry Palm	50	78.1%	16
7	Ken Massey	46	71.9%	15
7	RPI (Palm)	46	71.9%	15
9	Sagarin	45	70.3%	13
9	RealTime RPI (RPI)	45	70.3%	13
9	RealTime RPI bracket	45	70.3%	13
12	Sportheory	43	67.2%	10
12	Sagarin Pure Points	43	67.2%	10
14	pilight points only	41	64.1%	8
14	pilight's field of 64	41	64.1%	8
16	Stephen Jens	39	60.9%	6
17	Sonny Moore	36	56.3%	5
18	RealTime RPI (Power)	31	48.4%	4
19	AP poll	29	45.3%	3
20	Warren Nolan	27	42.2%	2
21	ESPN/Coaches poll	22	34.4%	1

Metric 4: predicted teams correctly seeded (top 12 seeds)

Rank		No.	Pct.	pts.
1	Charlie Crème	26	54.2%	21
2	College Women's Hoops S-Factor	23	47.9%	20
2	College Women's Hoops official bracket	23	47.9%	20
4	Sagarin	22	45.8%	18
4	Ken Massey	22	45.8%	18
6	Jerry Palm	21	43.8%	16
6	Sagarin ELO Chess	21	43.8%	16
6	Stephen Jens	21	43.8%	16
6	Cody Kirkpatrick (talismanred.com)	21	43.8%	16
10	RPI (Palm)	19	39.6%	12
11	RealTime RPI (RPI)	18	37.5%	11
11	pilight points only	18	37.5%	11
13	Sportheory	17	35.4%	9
14	ESPN/Coaches poll	15	31.3%	8
15	Sonny Moore	14	29.2%	7
16	Sagarin Pure Points	13	27.1%	6
17	pilight's field of 64	12	25.0%	5
17	AP poll	12	25.0%	5
19	RealTime RPI bracket	11	22.9%	3
19	RealTime RPI (Power)	11	22.9%	3
21	Warren Nolan	8	16.7%	1

Metric 5: Predicted teams within one seed of actual result (top 12 seeds)

Rank		No.	Pct.	pts.
1	College Women's Hoops official bracket	42	87.5%	21
2	Charlie Crème	40	83.3%	20
2	Cody Kirkpatrick (talismanred.com)	40	83.3%	20
4	College Women's Hoops S-Factor	38	79.2%	18
4	Sagarin ELO Chess	38	79.2%	18
6	Jerry Palm	36	75.0%	16
7	Sagarin	33	68.8%	15
7	Ken Massey	33	68.8%	15
7	RPI (Palm)	33	68.8%	15
7	RealTime RPI (RPI)	33	68.8%	15
7	RealTime RPI bracket	33	68.8%	15
12	Sportheory	32	66.7%	10
13	pilight points only	31	64.6%	9
13	pilight's field of 64	31	64.6%	9
15	Sagarin Pure Points	30	62.5%	7
16	AP poll	29	60.4%	6
17	Stephen Jens	28	58.3%	5
18	Sonny Moore	27	56.3%	4
19	ESPN/Coaches poll	22	45.8%	3
20	RealTime RPI (Power)	19	39.6%	2
21	Warren Nolan	18	37.5%	1

Metric 6: predicted teams correctly seeded (top 6 seeds)

Rank		No.	Pct.	pts.
1	College Women's Hoops official bracket	17	70.8%	21
1	Charlie Crème	17	70.8%	21
3	Ken Massey	16	66.7%	19
3	Stephen Jens	16	66.7%	19
5	Sagarin	15	62.5%	17
5	RPI (Palm)	15	62.5%	17
7	Sagarin ELO Chess	14	58.3%	15
7	Jerry Palm	14	58.3%	15
9	ESPN/Coaches poll	13	54.2%	13
10	College Women's Hoops S-Factor	12	50.0%	12
10	Sonny Moore	12	50.0%	12
12	RealTime RPI (RPI)	11	45.8%	10
12	Sportheory	11	45.8%	10
12	AP poll	11	45.8%	10
15	Cody Kirkpatrick (talismanred.com)	10	41.7%	7
15	pilight points only	10	41.7%	7
15	pilight's field of 64	10	41.7%	7
15	RealTime RPI (Power)	10	41.7%	7
19	RealTime RPI bracket	8	33.3%	3
19	Sagarin Pure Points	8	33.3%	3
21	Warren Nolan	7	29.2%	1

Metric 7: predicted teams seeded within one seed of actual result (top 6 seeds)

Rank		No.	Pct.	pts.
1	College Women's Hoops official bracket	24	100.0%	21
1	Charlie Crème	24	100.0%	21
3	Sagarin ELO Chess	23	95.8%	19
3	Jerry Palm	23	95.8%	19
3	AP poll	23	95.8%	19
6	College Women's Hoops S-Factor	22	91.7%	16
6	RealTime RPI (RPI)	22	91.7%	16
6	Cody Kirkpatrick (talismanred.com)	22	91.7%	16
9	RPI (Palm)	21	87.5%	13
9	Sportheory	21	87.5%	13
11	pilight points only	20	83.3%	11
11	pilight's field of 64	20	83.3%	11
11	RealTime RPI bracket	20	83.3%	11
14	Ken Massey	19	79.2%	8
14	Sagarin	19	79.2%	8
16	Stephen Jens	18	75.0%	6
16	ESPN/Coaches poll	18	75.0%	6
18	RealTime RPI (Power)	17	70.8%	4
18	Sagarin Pure Points	17	70.8%	4
20	Sonny Moore	16	66.7%	2
21	Warren Nolan	14	58.3%	1

______________________________

The overall winner? My bracket guess! Bested Charlie Creme by 3 points.

And once again, the College Women's Hoops S-Factor beats all other computer predictions.

Bracketology-ology Final Summary

rank	metric no.--->	1	2	3	4	5	6	7		sum of pts
1	College Women's Hoops official bracket	20	19	21	20	21	21	21		143
2	Charlie Crème	17	20	20	21	20	21	21		140
3	College Women's Hoops S-Factor	17	21	18	20	18	12	16		122
4	Jerry Palm	20	17	16	16	16	15	19		119
5	Sagarin ELO Chess	12	17	18	16	18	15	19		115
6	Ken Massey	17	17	15	18	15	19	8		109
7	RPI (Palm)	20	12	15	12	15	17	13		104
8	Cody Kirkpatrick (talismanred.com)	12	13	19	16	20	7	16		103
9	Sagarin	12	19	13	18	15	17	8		102
10	RealTime RPI (RPI)	21	12	13	11	15	10	16		98
11	Stephen Jens	8	17	6	16	5	19	6		77
12	pilight points only	17	10	8	11	9	7	11		73
13	Sportheory	8	9	10	9	10	10	13		69
14	RealTime RPI bracket	17	5	13	3	15	3	11		67
15	pilight's field of 64	12	6	8	5	9	7	11		58
16	Sonny Moore	8	8	5	7	4	12	2		46
17	AP poll	0	2	3	5	6	10	19		45
18	Sagarin Pure Points	5	7	10	6	7	3	4		42
19	ESPN/Coaches poll	0	5	1	8	3	13	6		36
20	RealTime RPI (Power)	5	5	4	3	2	7	4		30
21	Warren Nolan	3	2	2	1	1	1	1		11