Last Modified: 2011 October 9
Joe Schlobotnik / joe@amurgsval.org
System developed with Ken Butler, Gary Hatfield and Bob Stagat
URL for this frameset: http://slack.net/~whelan/tbrw/tbrw.cgi?2000/kpairwise.1.shtml
Since the 1995-1996 season, the NCAA has been using a system of pairwise comparisons to choose teams for the Men's Division I Ice Hockey Tournament. All eligible teams which finish the season with non-losing records against other eligible teams, or which receive automatic bids for winning conference championships, are considered teams under consideration (TUCs). Each pair of TUCs is compared using five criteria: Ratings Percentage Index (RPI, a linear combination of a team's winning percentage and the winning percentages of their opponents and their opponents' opponents), record against other TUCs, record in recent games, record against common opponents, and head-to-head games.
With a number of schools in recent years starting Division I programs or moving from Division I to Division II, and the formation of two new Division I leagues (the Metro-Atlantic Athletic Conference (MAAC) and College Hockey America (CHA)), which has allowed previously ineligible Division I programs to play enough D1 games to factor into tournament calculations, shortcomings of the present criteria have come to light. Notably, the RPI considers strength of schedule by including the average winning percentage of a team's opponents into the rating. However, if most or all of the teams in a conference are substantially weaker than those in other conferences, this fact will not be strongly reflected in their winning percentages, as they will play most of their games against one another. In addition, the "record vs TUCs" and "record in recent games" criteria do not consider the strength of the opposition at all. These flaws have had the effect of inflating the performance of two-time MAAC regular season champion Quinnipiac in pairwise comparisons. The Braves have been among the top twelve teams in the nation as measured by the selection criteria in the past two seasons, a fact which is widely attributed to maintaining a high winning percentage against a weak schedule. The Tournament Selection Committee has handled this situation by factoring the overall strength of a conference into its decisions in an unspecified way, and thereby leaving Quinnipiac out of the tournament field due to the weakness of the MAAC. This season, the weakness of College Hockey America was taken into account not by excluding CHA champion Niagara from the tournament, but by giving them a lower seed than their pairwise comparisons would have indicated.
These actions appear to be treating the symptoms rather than the disease. If the Ratings Percentage Index and other selection criteria are obviously inaccurate when judging members of predominantly weak conferences, it's reasonable to expect that they also misjudge members of establish conferences in ways that can have an impact on the seedings, but are not easily detected by looking at the strengths of the conferences as a whole. This has already happened with Niagara. It is fairly obvious, given the Purple Eagles' 18-7-2 record against a schedule of tournament-eligible competition which included several respectable opponents such as Boston University, St. Lawrence and Colorado College, but also five games against Air Force, that Niagara was easily among the better teams in Division I, but also clearly not the fifth or sixth best team in the nation, as their pairwise comparisons would imply. Without objective criteria to rely on, how did the committee decide that Niagara was worse than Colgate, New Hampshire, and Michigan State, but better than Minnesota State-Mankato, Rensselaer, and Colorado College? Would the decision to let Niagara into the tournament, but seed them at the bottom, have been altered if the Purple Eagles had won or lost a few more games? St. Cloud would in all likelihood have been left out of the tournament if they had lost to Minnesota in the WCHA consolation game, because this would have been enough to have cost them the pairwise comparison with the Gophers, but Niagara seems destined to have been the lowest-seeded team in the tournament regardless of the details of their results.