2013 Bracketology-ology

How did everyone do?  Depends on methodology, as always.

I analyzed 7 bracket predictions and 21 ranking models (listed in this post).  In 2011 I used seven metrics to judge which predictions were the best, and I have chosen these metrics once again this year:

•  Number of teams that were correctly selected
•  Of the entire field, number of teams that were correctly seeded
•  Of the entire field, number of teams seeded within one seed of the actual seed
•  Of the top 12 seeds (roughly the at-large cutoff), number of teams correctly seeded
•  Of the top 12 seeds, number of teams seeded within one
•  Of the top 6 seeds (roughly the AP and Coaches poll predictive cutoff), number of teams correctly seeded
•  Of the top 6 seeds, number of teams seeded within one

Here are the lists.

1. Correctly selected



2. Correctly seeded (entire field)



3. Seeded within 1 (entire field)



4. Correctly seeded (top 12 seeds)



5. Seeded within 1 (top 12 seeds)



6. Correctly seeded (top 6 seeds)



7. Seeded within 1 (top 6 seeds)



In 2011 I used a Borda count method (assigning a point value to a ranking and summing the points, like how the AP poll does it) to combine rankings for all seven metrics.  In 2011 the S-Factor was third best at prediction.  The S-Factor was a bit worse this year, coming in fifth, tied with pilight's Field of 64 method.   Still, by the Borda count method, the S-Factor remains the best numeric-only method at tournament prediction.  The bracket produced by RealtimeRPI.com takes top marks by this method.




This year I calculated a ranking based on Paymon points, which is how the Bracket Project analyzes the men's NCAA tournament bracketologists.  Paymon's method gives three points to each team selected, two points to each team correctly seeded, and one point to each team seeded within one seed of the predicted seed.  By the Paymon method, S-Factor comes in fourth among all methods, and is still the best numeric-only method.  Charlie Creme's bracket powers to victory by this method, as no other method or bracket came close to the number of correctly seeded teams.




A third way of comparing tournament predictions is to simply add up the number of correct picks falling into the above seven metrics; unlike in the Paymon method, this method captures the complexity and relative importance of teams seeded 1-6 and 7-12 over the teams seeded 13-16.  Another way of expressing this method, in formula format:

Where

• x_i  is 2 if predicted team i’s seed matches actual team i’s seed, 1 if predicted team i is seeded within 1, and 0 if predicted team i is two or more seeds off;
• y_i  is 3 if actual team i’s seed is 1 through 6, 2 if actual team i’s seed is 7 through 12, and 1 if actual team i’s seed is 13 through 16;
• z_i is 1 if actual team i was predicted in the tournament, 0 otherwise.

Charlie Creme's bracket is once again the best by this method.  S-Factor comes in sixth, ahead of all other numeric-only methods.  



  


See the data used for this post here! If you catch an error, let me know.

Who Got Screwed?

As part of my annual bracketology-ology summary of which women's tournament predictions were the best (coming soon), I analyzed seven bracket predictions and 21 team ranking systems.  They are as follows:

Brackets:
1. Charlie Creme's bracketology
2. My bracket
3. RealtimeRPI's predicted bracket
4. Omni Rankings - "Macro" bracket
5. Omni Rankings - "Micro" bracket
6. "The Field of 64" - pilight's bracket on Rebkell.
7. Matt5762's bracket on Rebkell.

Ranking systems:
8. S-Factor
9. WBBState's "The State"
10. RealtimeRPI power rankings
11. Sagarin ratings
12. Sagarin ELO CHESS only
13. Sagarin PURE POINTS only
14. Massey ratings
15. Massey's power ratings
16. Omni "Macro"
17. Omni "Micro"
18. Warren Nolan's "NPI"
19. SporTheory's rankings
20. Sonny Moore's Computer Power Ratings
21. CJBratings.com composite ratings
22. CJBratings.com "Win ratings"
23. Stats.com "UPS Team Performance Index"
24. RPI calculated by NCAA
25. RPI calculated by RealtimeRPI
26. Win-loss percentage (from NCAA.org)
27. AP poll
28. Coaches poll

For the ranking systems, I created a list of 64 teams that would have been predicted as being in the tournament by that system; that is, the top 33 at-large teams along with the 31 automatic bids.

I averaged the seed that would have been predicted by each bracket or ranking system.  Then I took the difference between the average predicted seed and the actual seed handed out by the selection committee.  The following is the list of all the tournament teams ranked in order from most screwed by the committee to least screwed.



By this method, Green Bay is the most screwed team in America. No ranking system or bracket had them below a 9 seed, yet they received an 11 seed.  Gonzaga too was drastically underseeded.

Others toward the top of this list (Albany, Quinnipiac) have a high average due to some out-there ranking systems, namely WBBState's "The State", Warren Nolan's NPI, Stats LLC's TPI, and the straight up W-L percentage.  So I made another list taking these four ranking systems out. 




In both lists Kansas is at the bottom of the list with a DIV/0 error. This is because not one prognostication selected Kansas into the field of 64. Kansas was ESPN's Charlie Creme's only selection error, the seventh team out in his rankings behind San Diego State, Charlotte, Toledo, Ohio State, Pacific, Florida Gulf Coast, and the team he had in in Kansas's stead, Duquesne.

The following is a list of at-large teams sorted by how many predictions would have selected them.



If the selection committee was run by the consensus of the internet, San Diego State, Michigan, West Virginia and Duquesne would have been the last teams in, and DePaul (surprisingly), Creighton, Ohio State and Charlotte the first four out. Kansas is at the bottom of the list, picked by no one to be a part of the tournament.

Final S-Factor of 2013

I ran the numbers one last time this morning. I have also made one last human-based bracket prediction.

In terms of 64-team selection, the one difference between the S-Factor and my bracket prediction is that I included West Virginia and left out Creighton.  I felt that to really be in contention for an at-large bid, Creighton needed to have made it to the title game of the Missouri Valley Conference tournament, and that didn't happen.  I think they'll be left out despite their relatively high RPI of 41.

What I'm not at all sure about is which team should replace them.  I put West Virginia in, but I would not be surprised if one of the regular-season-winning teams from weaker conferences (San Diego State, Florida Gulf Coast, Toledo) was included in the field of 64.  I know I wrote two weeks ago that Creighton and Toledo had decent chances to get into the tournament as at-large bids, but neither team made it to their conference tournament final. San Diego State and Florida Gulf Coast did make it to the final game, and so I think both of those teams stand a better chance.  But I put West Virginia in the field because they proved themselves capable on more than two occasions of beating top 50 teams (Iowa State, Oklahoma, Oklahoma State).

I feel that San Diego State, whose only win over a non-conference tournament-bound team was against Cal Poly, did not prove they could compete and win against the top teams. 

The team that makes me the most nervous though is Florida Gulf Coast.  They steamrolled through the Atlantic Sun conference, but they lost to Stetson in the tournament final. They have wins against tournament-bound LSU and Hampton, and 5 out of their 6 losses were against tournament-bound teams (assuming Charlotte gets in).  I could totally see the selection committee sticking FGCU in the field ahead of four other small conference regular season champs with better RPI ranking (Creighton (I know, co-champs), San Diego State, Pacific, Toledo).

I guess we'll see tonight.

I have Charlotte in the field despite their 53rd ranked RPI.  The S-Factor likes them even better than fellow Atlantic 10 conference mate Duquesne because of their 13-1 regular season record.  Charlie Creme puts West Virginia in the field instead of Charlotte.  Having both Duquesne and Charlotte in the field (in addition to Dayton and St. Joseph's) would be unusual: no more than three teams from the Atlantic 10 conference have been in the tournament since 1989. 

Comparisons to Charlie Creme's Bracketology

The penultimate run of the S-Factor changed little from Monday's run since most conference tournament games happened over the weekend.  It should be even less volatile for the last run, excepting Delaware, St. Joseph's, Green Bay, Creighton, and other teams from conferences whose tournaments have not yet been completed.

This is as good a time as any to point out the major differences between the S-Factor and Charlie Creme's bracketology, which has become the industry standard. 

S-Factor: Duquesne in, West Virginia second out
Creme: West Virginia in, Duquesne first out

The S-Factor favors Duquesne due to a better RPI ranking (41 to 52), fewer losses (7 to 13), and conference tournament wins (1 to 0). This outweighs West Virginia's better conference RPI and record against top 50 teams in my model. When weighted against conference RPI, 9-9 in the Big 12  is about equivalent to 11-3 in the Atlantic 10.


Purdue
S-Factor: 4-seed
Creme: 7-seed

5% of the S-Factor score comes from conference tournament performance. Winning the Big Ten tournament bumped Purdue from a 7-seed to a 4-seed in the S-Factor. Tournament games also bumped SEC-winning Texas A&M from a 5-seed to a 3-seed in the S-Factor, which is where Creme has them.

Michigan State
S-Factor: 5-seed
Creme: 8-seed

The S-Factor has been a little more bullish on Michigan State since February when they beat Purdue, a team high in RPI and low in respect.

Gonzaga
S-Factor: 7-seed
Creme: 11-seed

The West Coast Conference is in 7th place out of 31 conferences in terms of RPI, ahead of even the Atlantic 10. It has four teams in the top 75 in RPI and no teams with a strength of schedule in the bottom half of the country, which makes Gonzaga's current 15 game winning streak more impressive than similarly positioned teams like Delaware and Green Bay. That would be my pitch to the selection committee to give Gonzaga the 7-seed rather than a 10- or 11-seed (as a host school it is unlikely to get an 8- or 9-seed because that would involve a 1-seed playing an away game in the second round.).  

Green Bay
S-Factor: 10-seed
Creme: 7-seed

It's hard to feel confident in an evaluation of a team when that team plays few marquee games. An impressive 26-2 record is hedged by the fact that all we really know is that Green Bay is the best team in Wisconsin. 

Texas Tech
S-Factor: 10-seed
Creme: 6-seed

As much as conference RPI plays a role in the S-Factor (and therefore teams in the Big 12, the nations top RPI conference, get a slightly larger boost than other major conferences), a team's RPI  counts for more, as do top 25 and top 50 wins. The entire non-Baylor Big 12 has no non-Iowa State top 25 wins (Iowa State is 22nd in RPI but has been in and out of the top 25 all season).

Texas Tech doesn't even have an Iowa State win. The Red Raiders have a 48th ranked RPI, which is solidly in the  bubble team zone historically.  I'm not saying there's a chance Tech will miss the tournament (a Big 12 team missing the tournament after winning more than 60% of their games would be an outrage). I'm just saying that everything besides conference winning record (three straight losses to unranked opponents, no conference tournament wins, ten losses on the season, soft non-conference schedule coupled with an unimpressive performance, best win all season was against the Big 12's 5th place team) points to a 10-seed or worse (like Gonzaga, Tech is a host school unlikely to be an 8- or 9-seed even if their resume was up to that level)

Princeton
S-Factor: 12-seed
Creme: 9-seed

I'd like to say Princeton's downfall in the S-Factor was their loss to Harvard, the lone blemish against a dominating Ivy League performance, but it would be untrue: the S-Factor has never had Princeton higher than a 12 seed. Princeton's impressive 26th ranked RPI is offset by the Ivy League's unimpressive 18th (of 31) conference RPI. 

S-Factor and Duke: A Love Affair

Okay, so it can no longer be avoided: when the final model run takes place one week from today, the S-Factor is going to end up showing Duke as a number one-seed. Either Connecticut or Stanford, almost unanimously shown as one seeds by other bracketology sites, is going to be shown as a two-seed.  Connecticut will be no lower than #4 overall if they win the Big East tournament, but if they lose in the championship game or before, they will almost certainly be shown where they are right now, in the #5 overall spot.

I outlined the case for Duke back in February. Since then, two bracket-changing events have happened: Duke lost to 50th-ranked-in-RPI Miami, and Duke killed it in the ACC tournament, triumphing in the championship by 19 points over North Carolina.  On the one hand, that loss to Miami is the only loss by any of the top 5 teams (Baylor, Stanford, Notre Dame, Connecticut, Duke) to a team outside the top 25. On the other hand, Duke won both the regular season and the conference tournament of the ACC, and they went 5-0 against top 25 teams since Chelsea Gray's injury.

Still, Connecticut and Stanford seem like objectively better teams.  They have losses only to teams that will be #1 or #2 seeds in the tournament, and they both have more blowouts of 20+ points (Connecticut 24, Stanford 18, Duke 16, none since Chelsea Gray's injury).  Connecticut at the very least should definitely be ahead of Duke, if for no other reason than the 30 point blowout they handed to Duke.

Now in:
St. John's
UT Martin
Seattle
Stetson
Navy


Now out:
Illinois
Eastern Illinois
Utah State
Florida Gulf Coast
Army

Conferences with multiple bids:
Big East: 8
SEC: 7
Big Ten: 6
Big 12: 5
ACC: 5
Pac 12: 4
Atlantic 10: 4
Missouri Valley: 2

Last Notes Before the Conference Tournaments

 - The Big 12 regular season wrapped up on Tuesday, and the conference tournament starts tomorrow in Dallas.  After West Virginia's unexpected loss to Texas on Tuesday, for the first time all season, the S-Factor is showing only five Big 12 teams to make the tournament: Baylor (1), Iowa State (5), Oklahoma (6), Texas Tech (8), and Oklahoma State (11).

 - Backing into West Virginia's place in the tournament for now is Illinois, but St. John's is a much more likely candidate for a slot in the field. With an impressive 11-5 record in the Big East, all but three of their 11 losses coming against NCAA tournament teams, 34th ranked RPI, and an important first round hosting slot, St. John's is pretty much a no-brainer for a tournament bid (an anomalous loss to RPI 203 Seton Hall is currently costing them about 5-6 S-Factor points, which would have put them somewhere around an 8-seed in the S-Factor had they won). 

 - I think Illinois, South Florida, St. Joseph's, and Oklahoma State are the bubbliest teams right now that I have shown in the field.



 - Conferences where the conference tournament has little chance of altering the selection of at-large bids: SEC, ACC, Pac-12.  The SEC is very likely getting 7 teams in: there is a huge gap between 7th place Vanderbilt (9-7 SEC) and 8th place Arkansas (6-10 SEC).  Likewise between 5th place Miami (11-7 ACC) and 6th place Virginia (8-10 ACC), and between 4th place Colorado (13-5 Pac-12, 23 RPI) and 5th place Washington (11-7 Pac-12, 83 RPI). 

Now in:
Illinois

Now out:
West Virginia

Conferences with multiple bids:
Big Ten: 7
Big East: 7
SEC: 7
Big 12: 5
ACC: 5
Pac-12: 4
Atlantic 10: 4
Missouri Valley: 2

Bubble Poppers: At-Large Bids from Small Conferences

The conference regular season has now wrapped up in 15 out of the 31 automatic-bid-producing conferences, and conference tournaments begin this week in some of those 15.  With conference tournaments comes the chance that a top-seeded team will lose and forfeit the automatic bid that bracketologists have bestowed on it for so long.  For teams on the bubble, this may be bad news if the losing team is good enough to qualify for an at-large bid. One less slot for the South Floridas of the nation.

Here are 10 teams that could pop some other team's bubble if they lose in their conference tournament:

1. Delaware (26-3, 17-0 Colonial)
16th ranked in the latest AP poll and with a winning streak dating back to 2012, the Blue Hens are not missing the tournament this year.  But if the unthinkable happens and they lose in the Colonial tournament, the CAA will get two teams in the field when they would have only gotten one in otherwise.

2. Green Bay (24-2, 14-0)
20th ranked Green Bay has only two losses this season, none since December 5. Though their strength of schedule is abysmal, their consistency will likely be rewarded by the selection committee, even if they lose.

3. Gonzaga (25-5, 15-1 WCC)
Gonzaga started the conference season as one of three or four contenders for the West Coast Conference regular season crown. Since their loss to St. Mary's on January 10, the Bulldogs have gone undefeated while their conference contenders (BYU, St. Mary's, San Diego) sort of fell off, making the West Coast Conference a one-bid conference, unless Gonzaga loses.

4. Creighton (21-6, 13-3 Missouri Valley)
Creighton is the only team on this list that the S-Factor does not currently show as an automatic bid, but as an at-large bid.  Their RPI (26th ranked) is impressive, but their selection as an at-large bid is by no means certain given how rough the Valley's regular season was to them.  Still, wins against Nebraska and South Florida continue to look good for the Blue Jays, and I think they will have a good chance at getting an at-large bid. 

5. Toledo (26-2, 14-1 MAC)
With all the wild results from this weekend and Thursday, Toledo slipped into the AP rankings this week to #25.  Toledo is like a light-version of Green Bay: almost as dominant in conference (just one loss) almost as lengthy of a winning streak (last loss on January 10), and a strength of schedule that is slightly worse (the 222nd most difficult in the nation).  Should they lose, the Rockets still stand a good chance at getting an at-large bid. Maybe.

6. San Diego State (22-5, 13-1 MWC)
The Mountain West Conference lost a lot of its luster when TCU and Utah left, which is why the MWC is only getting one team in the field of 64 if San Diego State wins the tournament.  The MWC tournament though is notorious for upending the top seed, which has lost in six of the last seven conference tournaments.   San Diego State's last loss was on January 9, and their RPI of 36 may be enough to get them into the NCAA tournament without having to win the MWC tournament. Maybe.

7. Chattanooga (26-3, 19-1 SoCon)
Chattanooga had that monster win against Tennessee to open the season but lost 3 games against beatable opponents. Their 46-ranked RPI will probably keep them from the big dance if they lose the Southern Conference tournament. Maybe.

8. Florida Gulf Coast (25-5, 18-0 Atlantic Sun)
Florida Gulf Coast, still in only their second NCAA tournament-eligible season, dominated the Atlantic Sun Conference, and they went 2-3 against the big six conferences, including a win against LSU that is looking better and better as the season progresses. Still, the Atlantic Sun is a terrible conference (28th of 31 in conference RPI terms) and two bids from it seems unlikely.

9. Quinnipiac (26-2, 17-0 NEC)
The unexpected dominance of the Northeast Conference by Quinnipiac  this year might be enough to earn the Bobcats a bid to the NCAA tournament if they lose in the conference tournament. But like Toledo, their SOS is abysmal, and they'd basically be relying on a 3-point win against St. John's to get them in should they lose the NEC tournament.

10. Marist (23-6, 18-0 MAAC)
Though it hasn't happened in eight years, we must at least assume that Marist failing to get the automatic bid in the MAAC (Marist, Automatic, Always! Conference) is a possibility.  If that happens, could they get into the NCAA tournament anyways? I say it's unlikely given their six losses, their 49th ranked RPI, and their best win being over Princeton.