Metabracketology 2015

This is my annual post about metabracketology, or bracketology-ology.

The College Women's Hoops S-Factor correctly predicted 62 of the 64 teams in the tournament this year, missing on the inclusion of Tulane and Arkansas.  The S-Factor predicted the seeds of the field within one seed of the result for 51 of the 64 teams, far better than the next nearest numeric ranking algorithm (Omni Rankings, only 43 of 64).

I looked at all 33 of the prediction methodologies included in the Who Got Screwed? post, plus the ranking method of WayneBearCal on rebkell.net, which is a composite of the Massey Rating, Massey Power Rating, RPI, and Sagarin PurePoints Predictor.

 

Table 1: Selection of entire field

Rank	Overall selection	No.	Pct.	Borda count pts.	Paymon pts.
1	Creme's bracket	64	100%	34	192
1	Creme, no bumps	64	100%	34	192
3	SpaceJunkie	63	98%	32	189
3	Matt5762 bracket	63	98%	32	189
3	CSM bracket	63	98%	32	189
6	S-Factor (2015 formula)	62	97%	29	186
6	My bracket	62	97%	29	186
6	WayneBearCal "Proper"	62	97%	29	186
6	S-Factor (old blend)	62	97%	29	186
6	RealtimeRPI RPI	62	97%	29	186
6	RealtimeRPI bracket	62	97%	29	186
6	Pilight's Field of 64	62	97%	29	186
6	Massey Rating	62	97%	29	186
6	Adjusted Stats EWP	62	97%	29	186
15	Warren Nolan RPI	61	95%	20	183
15	Sagarin Rating	61	95%	20	183
15	Sagarin ELO	61	95%	20	183
15	NCAA RPI	61	95%	20	183
15	Massey Power	61	95%	20	183
15	GaTech LRMC(0)	61	95%	20	183
15	BennettRank	61	95%	20	183
22	Sonny Moore	60	94%	13	180
22	Sagarin PURE_POINTS	60	94%	13	180
22	RealtimeRPI Power	60	94%	13	180
22	Omni Rankings	60	94%	13	180
26	Omni bracket	59	92%	9	177
26	GaTech LRMC Classic	59	92%	9	177
26	GaTech LRMC Bayesian	59	92%	9	177
29	Sagarin GoldenMean	58	91%	6	174
30	Warren Nolan NPI	55	86%	5	165
31	NCAA W-L %	51	80%	4	153
32	D1SN "poll"	46	72%	3	138
33	AP poll	36	56%	2	108
34	Coaches poll	35	55%	1	105

Table 2: Predicted exact seeds, entire field of 64

Rank	Exact seeds – entire field	No.	Pct.	Borda count pts.	Paymon pts.
1	SpaceJunkie	30	46.9%	34	60
1	Matt5762 bracket	30	46.9%	34	60
1	Creme's bracket	30	46.9%	34	60
4	CSM bracket	29	45.3%	31	58
4	Creme, no bumps	29	45.3%	31	58
6	Omni Rankings	28	43.8%	29	56
7	My bracket	25	39.1%	28	50
7	Sagarin PURE_POINTS	25	39.1%	28	50
7	GaTech LRMC Classic	25	39.1%	28	50
7	BennettRank	25	39.1%	28	50
11	Adjusted Stats EWP	24	37.5%	24	48
12	WayneBearCal "Proper"	23	35.9%	23	46
12	Massey Power	23	35.9%	23	46
14	S-Factor (2015 formula)	22	34.4%	21	44
14	Sonny Moore	22	34.4%	21	44
14	Sagarin Rating	22	34.4%	21	44
14	Omni bracket	22	34.4%	21	44
18	S-Factor (old blend)	21	32.8%	17	42
18	Massey Rating	21	32.8%	17	42
20	RealtimeRPI bracket	20	31.3%	15	40
20	GaTech LRMC Bayesian	20	31.3%	15	40
22	Sagarin ELO	17	26.6%	13	34
22	Pilight's Field of 64	17	26.6%	13	34
24	Sagarin GoldenMean	16	25.0%	11	32
24	RealtimeRPI Power	16	25.0%	11	32
24	D1SN "poll"	16	25.0%	11	32
27	RealtimeRPI RPI	13	20.3%	8	26
27	GaTech LRMC(0)	13	20.3%	8	26
27	AP poll	13	20.3%	8	26
30	Coaches poll	12	18.8%	5	24
31	NCAA RPI	10	15.6%	4	20
32	Warren Nolan RPI	9	14.1%	3	18
33	Warren Nolan NPI	8	12.5%	2	16
34	NCAA W-L %	7	10.9%	1	14

The S-Factor and the RPI calculations predicted exact seeds shockingly poorly this year. Last year, RPI correctly predicted the seeds of 29 teams, second only to Charlie Creme. This year, RPI only predicted the seeds of 9 teams, good for 31st place. The S-Factor was better, having predicted the seeds of 22 teams correctly, but this was a middle-of-the-pack result.

Table 3: Predicted seeds within 1 seed of actual result, entire field

Rank	Seeds within 1 - entire field	No.	Pct.	Borda count pts.
1	SpaceJunkie	57	89.1%	34
2	Creme, no bumps	55	85.9%	33
3	Matt5762 bracket	54	84.4%	32
3	Creme's bracket	54	84.4%	32
5	S-Factor (2015 formula)	51	79.7%	30
5	RealtimeRPI bracket	51	79.7%	30
7	CSM bracket	50	78.1%	28
8	Omni Rankings	43	67.2%	27
8	Omni bracket	43	67.2%	27
8	BennettRank	43	67.2%	27
11	WayneBearCal "Proper"	42	65.6%	24
11	Sagarin Rating	42	65.6%	24
11	Sagarin PURE_POINTS	42	65.6%	24
14	My bracket	41	64.1%	21
15	S-Factor (old blend)	40	62.5%	20
15	Sagarin GoldenMean	40	62.5%	20
15	Massey Power	40	62.5%	20
18	Sagarin ELO	39	60.9%	17
18	RealtimeRPI Power	39	60.9%	17
18	Pilight's Field of 64	39	60.9%	17
18	Massey Rating	39	60.9%	17
18	GaTech LRMC(0)	39	60.9%	17
23	Warren Nolan RPI	38	59.4%	12
23	NCAA RPI	38	59.4%	12
23	GaTech LRMC Classic	38	59.4%	12
23	Adjusted Stats EWP	38	59.4%	12
27	Sonny Moore	37	57.8%	8
27	GaTech LRMC Bayesian	37	57.8%	8
29	RealtimeRPI RPI	36	56.3%	6
30	D1SN "poll"	28	43.8%	5
31	Coaches poll	24	37.5%	4
32	AP poll	22	34.4%	3
33	Warren Nolan NPI	20	31.3%	2
34	NCAA W-L %	16	25.0%	1

The S-Factor was a much better predictor of teams once the window was widened to include seeds plus or minus one from predicted.  80% of teams were seeded nearly correctly by the S-Factor.  Of course, we were all eating SpaceJunkie's dust this year.

Table 4: Predicted exact seeds, top 12 seeds only

Rank	Exact seeds - Top 12 seeds only	No.	Pct.	Borda count pts.
1	SpaceJunkie	27	56.3%	34
1	Matt5762 bracket	27	56.3%	34
3	Creme's bracket	26	54.2%	32
4	Creme, no bumps	25	52.1%	31
5	CSM bracket	23	47.9%	30
6	BennettRank	20	41.7%	29
7	My bracket	19	39.6%	28
7	Omni Rankings	19	39.6%	28
7	Adjusted Stats EWP	19	39.6%	28
10	GaTech LRMC Classic	18	37.5%	25
11	S-Factor (2015 formula)	17	35.4%	24
11	Sagarin PURE_POINTS	17	35.4%	24
13	WayneBearCal "Proper"	16	33.3%	22
13	S-Factor (old blend)	16	33.3%	22
13	Massey Rating	16	33.3%	22
13	Massey Power	16	33.3%	22
13	D1SN "poll"	16	33.3%	22
18	Sonny Moore	15	31.3%	17
18	Sagarin Rating	15	31.3%	17
18	Omni bracket	15	31.3%	17
18	GaTech LRMC Bayesian	15	31.3%	17
22	Sagarin ELO	14	29.2%	13
22	RealtimeRPI bracket	14	29.2%	13
24	RealtimeRPI Power	13	27.1%	11
24	AP poll	13	27.1%	11
26	Pilight's Field of 64	12	25.0%	9
26	Coaches poll	12	25.0%	9
28	Sagarin GoldenMean	11	22.9%	7
28	GaTech LRMC(0)	11	22.9%	7
30	RealtimeRPI RPI	10	20.8%	5
31	Warren Nolan RPI	7	14.6%	4
31	NCAA RPI	7	14.6%	4
33	Warren Nolan NPI	5	10.4%	2
34	NCAA W-L %	4	8.3%	1

These tables show predictions for the top 12 seeds only, since these are the seeds of the at-large bids.

Table 5: Predicted seeds within 1 seed of actual result, top 12 seeds only

Rank	Seeds within one - Top 12 seeds only	No.	Pct.	Borda count pts.
1	SpaceJunkie	41	85.4%	34
1	Matt5762 bracket	41	85.4%	34
3	Creme's bracket	40	83.3%	32
4	Creme, no bumps	39	81.3%	31
5	S-Factor (2015 formula)	38	79.2%	30
5	My bracket	38	79.2%	30
5	RealtimeRPI bracket	38	79.2%	30
8	CSM bracket	36	75.0%	27
9	WayneBearCal "Proper"	30	62.5%	26
9	Omni Rankings	30	62.5%	26
9	BennettRank	30	62.5%	26
12	S-Factor (old blend)	29	60.4%	23
12	Omni bracket	29	60.4%	23
14	Sagarin Rating	28	58.3%	21
14	Sagarin PURE_POINTS	28	58.3%	21
14	Massey Power	28	58.3%	21
14	D1SN "poll"	28	58.3%	21
18	Pilight's Field of 64	27	56.3%	17
18	Massey Rating	27	56.3%	17
20	Sagarin GoldenMean	26	54.2%	15
20	Sagarin ELO	26	54.2%	15
20	RealtimeRPI Power	26	54.2%	15
23	GaTech LRMC(0)	25	52.1%	12
23	GaTech LRMC Bayesian	25	52.1%	12
23	Adjusted Stats EWP	25	52.1%	12
26	Warren Nolan RPI	24	50.0%	9
26	Sonny Moore	24	50.0%	9
26	RealtimeRPI RPI	24	50.0%	9
26	NCAA RPI	24	50.0%	9
26	GaTech LRMC Classic	24	50.0%	9
26	Coaches poll	24	50.0%	9
32	AP poll	22	45.8%	3
33	Warren Nolan NPI	11	22.9%	2
34	NCAA W-L %	8	16.7%	1

Table 6: predicted exact seeds, top 6 seeds only

Rank	Exact seeds - Top 6 seeds only	No.	Pct.	Borda count pts.
1	CSM bracket	18	75.0%	34
2	Creme's bracket	16	66.7%	33
2	Creme, no bumps	16	66.7%	33
4	Adjusted Stats EWP	15	62.5%	31
5	SpaceJunkie	14	58.3%	30
5	Massey Power	14	58.3%	30
7	GaTech LRMC Classic	13	54.2%	28
7	D1SN "poll"	13	54.2%	28
7	BennettRank	13	54.2%	28
7	AP poll	13	54.2%	28
11	My bracket	12	50.0%	24
11	WayneBearCal "Proper"	12	50.0%	24
11	Sagarin Rating	12	50.0%	24
11	Sagarin PURE_POINTS	12	50.0%	24
11	RealtimeRPI Power	12	50.0%	24
11	Omni Rankings	12	50.0%	24
11	Matt5762 bracket	12	50.0%	24
11	GaTech LRMC Bayesian	12	50.0%	24
19	S-Factor (old blend)	11	45.8%	16
19	RealtimeRPI bracket	11	45.8%	16
19	Massey Rating	11	45.8%	16
22	Sonny Moore	10	41.7%	13
22	Omni bracket	10	41.7%	13
22	Coaches poll	10	41.7%	13
25	Sagarin GoldenMean	9	37.5%	10
25	Sagarin ELO	9	37.5%	10
27	S-Factor (2015 formula)	8	33.3%	8
27	Pilight's Field of 64	8	33.3%	8
27	GaTech LRMC(0)	8	33.3%	8
30	Warren Nolan RPI	6	25.0%	5
30	RealtimeRPI RPI	6	25.0%	5
30	NCAA RPI	6	25.0%	5
33	Warren Nolan NPI	5	20.8%	2
34	NCAA W-L %	4	16.7%	1

These two tables exist to accurately compare ranked lists to the AP and Coaches' polls. The AP poll predicted more exact seeds, but the Coaches' poll predicted more seeds within 1.

Table 7: predicted seeds within 1 seed of actual result, top 6 seeds only

Rank	Seeds within 1 - Top 6 seeds only	No.	Pct.	Borda count pts.
1	Matt5762 bracket	23	95.8%	34
1	CSM bracket	23	95.8%	34
3	Creme's bracket	22	91.7%	32
4	S-Factor (2015 formula)	21	87.5%	31
4	My bracket	21	87.5%	31
4	SpaceJunkie	21	87.5%	31
4	Creme, no bumps	21	87.5%	31
8	GaTech LRMC(0)	20	83.3%	27
8	Coaches poll	20	83.3%	27
10	S-Factor (old blend)	19	79.2%	25
10	RealtimeRPI Power	19	79.2%	25
10	RealtimeRPI bracket	19	79.2%	25
10	D1SN "poll"	19	79.2%	25
14	Sagarin Rating	18	75.0%	21
14	Omni Rankings	18	75.0%	21
14	Omni bracket	18	75.0%	21
14	BennettRank	18	75.0%	21
14	AP poll	18	75.0%	21
19	WayneBearCal "Proper"	17	70.8%	16
19	Sagarin PURE_POINTS	17	70.8%	16
19	Sagarin GoldenMean	17	70.8%	16
19	GaTech LRMC Bayesian	17	70.8%	16
19	Adjusted Stats EWP	17	70.8%	16
24	Warren Nolan RPI	16	66.7%	11
24	RealtimeRPI RPI	16	66.7%	11
24	Pilight's Field of 64	16	66.7%	11
24	NCAA RPI	16	66.7%	11
24	Massey Rating	16	66.7%	11
24	Massey Power	16	66.7%	11
24	GaTech LRMC Classic	16	66.7%	11
31	Sonny Moore	15	62.5%	4
32	Sagarin ELO	14	58.3%	3
33	Warren Nolan NPI	9	37.5%	2
34	NCAA W-L %	7	29.2%	1

We had some excellent predictions this year over on the old rebkell.net boards. Though Charlie Creme of ESPN.com correctly picked all 64 teams this year as usual, the winner by all three scoring metrics (tied with Creme in the Borda and Paymon methods for 1st, outright winner on the Sum of Points method) was a user submitted bracket from SpaceJunkie.

BennettRank edged out the S-Factor for best numeric only method by the Borda count method (the way AP tallies its rankings).

Borda count method

	Borda count points	1	2	3	4	5	6	7	Sum
1	SpaceJunkie	32	34	34	34	34	30	31	229
1	Creme's bracket	34	34	32	32	32	33	32	229
3	Matt5762 bracket	32	34	32	34	34	24	34	224
3	Creme, no bumps	34	31	33	31	31	33	31	224
5	CSM bracket	32	31	28	30	27	34	34	216
6	My bracket	29	28	21	28	30	24	31	191
7	BennettRank	20	28	27	29	26	28	21	179
8	S-Factor (2015 formula)	29	21	30	24	30	8	31	173
9	Omni Rankings	13	29	27	28	26	24	21	168
10	WayneBearCal "Proper"	29	23	24	22	26	24	16	164
11	RealtimeRPI bracket	29	15	30	13	30	16	25	158
12	S-Factor (old blend)	29	17	20	22	23	16	25	152
12	Adjusted Stats EWP	29	24	12	28	12	31	16	152
14	Sagarin PURE_POINTS	13	28	24	24	21	24	16	150
15	Sagarin Rating	20	21	24	17	21	24	21	148
16	Massey Power	20	23	20	22	21	30	11	147
17	Omni bracket	9	21	27	17	23	13	21	131
18	Massey Rating	29	17	17	22	17	16	11	129
19	GaTech LRMC Classic	9	28	12	25	9	28	11	122
20	RealtimeRPI Power	13	11	17	11	15	24	25	116
21	D1SN "poll"	3	11	5	22	21	28	25	115
22	Pilight's Field of 64	29	13	17	9	17	8	11	104
23	GaTech LRMC Bayesian	9	15	8	17	12	24	16	101
24	GaTech LRMC(0)	20	8	17	7	12	8	27	99
25	Sagarin ELO	20	13	17	13	15	10	3	91
26	Sonny Moore	13	21	8	17	9	13	4	85
26	Sagarin GoldenMean	6	11	20	7	15	10	16	85
28	AP poll	2	8	3	11	3	28	21	76
29	RealtimeRPI RPI	29	8	6	5	9	5	11	73
30	Coaches poll	1	5	4	9	9	13	27	68
31	NCAA RPI	20	4	12	4	9	5	11	65
32	Warren Nolan RPI	20	3	12	4	9	5	11	64
33	Warren Nolan NPI	5	2	2	2	2	2	2	17
34	NCAA W-L %	4	1	1	1	1	1	1	10

The S-Factor was the best numeric only method by the Sum of Points method. I described the Sum of Points method this way in 2013.

Another way of expressing this method, in formula format:



Where

• x_i  is 2 if predicted team i’s seed matches actual team i’s seed, 1 if predicted team i is seeded within 1, and 0 if predicted team i is two or more seeds off;

• y_i  is 3 if actual team i’s seed is 1 through 6, 2 if actual team i’s seed is 7 through 12, and 1 if actual team i’s seed is 13 through 16;

• z_i is 1 if actual team i was predicted in the tournament, 0 otherwise.

Baylor was generally believed to be the 1st place team that year and Prairie View A&M the 64th place team, so the sum is over all teams in the NCAA tournament.

Sum of points

	Sum of points	1	2	3	4	5	6	7	Sum
1	SpaceJunkie	63	30	57	27	41	14	21	253
2	Creme's bracket	64	30	54	26	40	16	22	252
3	Matt5762 bracket	63	30	54	27	41	12	23	250
4	Creme, no bumps	64	29	55	25	39	16	21	249
5	CSM bracket	63	29	50	23	36	18	23	242
6	S-Factor (2015 formula)	62	22	51	17	38	8	21	219
7	My bracket	62	25	41	19	38	12	21	218
8	RealtimeRPI bracket	62	20	51	14	38	11	19	215
9	Omni Rankings	60	28	43	19	30	12	18	210
9	BennettRank	61	25	43	20	30	13	18	210
11	WayneBearCal "Proper"	62	23	42	16	30	12	17	202
12	Sagarin PURE_POINTS	60	25	42	17	28	12	17	201
13	Adjusted Stats EWP	62	24	38	19	25	15	17	200
14	S-Factor (old blend)	62	21	40	16	29	11	19	198
14	Sagarin Rating	61	22	42	15	28	12	18	198
14	Massey Power	61	23	40	16	28	14	16	198
17	Omni bracket	59	22	43	15	29	10	18	196
18	GaTech LRMC Classic	59	25	38	18	24	13	16	193
19	Massey Rating	62	21	39	16	27	11	16	192
20	RealtimeRPI Power	60	16	39	13	26	12	19	185
20	GaTech LRMC Bayesian	59	20	37	15	25	12	17	185
22	Sonny Moore	60	22	37	15	24	10	15	183
23	Pilight's Field of 64	62	17	39	12	27	8	16	181
24	Sagarin ELO	61	17	39	14	26	9	14	180
25	Sagarin GoldenMean	58	16	40	11	26	9	17	177
25	GaTech LRMC(0)	61	13	39	11	25	8	20	177
27	RealtimeRPI RPI	62	13	36	10	24	6	16	167
28	D1SN "poll"	46	16	28	16	28	13	19	166
29	NCAA RPI	61	10	38	7	24	6	16	162
30	Warren Nolan RPI	61	9	38	7	24	6	16	161
31	Coaches poll	35	12	24	12	24	10	20	137
31	AP poll	36	13	22	13	22	13	18	137
33	Warren Nolan NPI	55	8	20	5	11	5	9	113
34	NCAA W-L %	51	7	16	4	8	4	7	97

The Bracket Matrix project analyzes bracket predictions for the men's game based on the Paymon method, which gives three points to every team selected in the field, two bonus points to every team with a correctly predicted seed, and one bonus point for every team predicted to be within 1 seed of the final selection.

SpaceJunkie and Charlie Creme tie for the best bracket prediction by this method. The S-Factor is the best numeric-only ranking algorithm by the Paymon method.

Paymon method

	Paymon method	1	2	3	Sum
1	SpaceJunkie	189	60	57	306
1	Creme's bracket	192	60	54	306
3	Creme, no bumps	192	58	55	305
4	Matt5762 bracket	189	60	54	303
5	CSM bracket	189	58	50	297
6	S-Factor (2015 formula)	186	44	51	281
7	Omni Rankings	180	56	43	279
8	RealtimeRPI bracket	186	40	51	277
8	My bracket	186	50	41	277
10	BennettRank	183	50	43	276
11	WayneBearCal "Proper"	186	46	42	274
12	Sagarin PURE_POINTS	180	50	42	272
12	Adjusted Stats EWP	186	48	38	272
14	Sagarin Rating	183	44	42	269
14	Massey Power	183	46	40	269
16	S-Factor (old blend)	186	42	40	268
17	Massey Rating	186	42	39	267
18	GaTech LRMC Classic	177	50	38	265
19	Omni bracket	177	44	43	264
20	Sonny Moore	180	44	37	261
21	Pilight's Field of 64	186	34	39	259
22	Sagarin ELO	183	34	39	256
23	GaTech LRMC Bayesian	177	40	37	254
24	RealtimeRPI Power	180	32	39	251
25	RealtimeRPI RPI	186	26	36	248
25	GaTech LRMC(0)	183	26	39	248
27	Sagarin GoldenMean	174	32	40	246
28	NCAA RPI	183	20	38	241
29	Warren Nolan RPI	183	18	38	239
30	Warren Nolan NPI	165	16	20	201
31	D1SN "poll"	138	32	28	198
32	NCAA W-L %	153	14	16	183
33	AP poll	108	26	22	156
34	Coaches poll	105	24	24	153