View source for Sports rating system

{{Short description|none}}
{{multiple issues|
{{Globalize|article|USA|2name=the United States|date=September 2016}}
{{broaden |date=September 2016}}
}}
A '''sports rating system''' is a system that analyzes the results of sports competitions to provide [[Evaluation|rating]]s for each team or player. Common systems include polls of expert voters, [[crowdsourcing]] non-expert voters, betting markets, and computer systems. Ratings, or '''power ratings''', are numerical representations of competitive strength, often directly comparable so that the game outcome between any two teams can be predicted. [[Ranking]]s, or '''power rankings''', can be directly provided (e.g., by asking people to rank teams), or can be derived by sorting each team's ratings and assigning an [[Ordinal scale|ordinal]] rank to each team, so that the highest rated team earns the #1 rank. Rating systems provide an alternative to traditional sports [[standings]] which are based on win–loss–tie ratios.

[[File:UCF at the Texas goal line.jpg|thumb|right|College football players in the United States]]
In the United States, the biggest use of sports ratings systems is to rate [[National Collegiate Athletic Association|NCAA]] college football teams in [[NCAA Division I|Division I FBS]], choosing teams to play in the [[College Football Playoff]]. Sports ratings systems are also used to help determine the field for the NCAA [[College basketball|men's and women's basketball]] tournaments, men's professional [[golf]] tournaments, professional [[tennis]] tournaments, and [[NASCAR]]. They are often mentioned in discussions about the teams that could or should receive invitations to participate in certain contests, despite not earning the most direct entrance path (such as a league championship).<ref>{{cite web | url=http://aol.sportingnews.com/ncaa-basketball/story/2011-03-09/sorting-through-teams-on-one-big-bubble |website=[[Sporting News]] |access-date=2011-03-24 |date=2011-03-09 |last=Fagan |first=Ryan |title=Sorting through teams on one big bubble |quote=This is a look at 20 of the teams (in alphabetical order) residing on this year’s big ol’ bubble. We’ve included three statistical rankings. The RPI (ratings percentage index, taken from collegeRPI.com) is considered the standard and is provided to committee members during the selection process. The two other ranking indexes include margin of victory in their formulas—the Pomeroy ratings (at kenpom.com) and Sagarin ratings (via USA Today)—aren’t new but have played an increased role in discussions about potential seeds during this college basketball season.}}</ref>

Computer rating systems can tend toward [[Objectivity (science)|objectivity]], without specific player, team, regional, or style bias. [[Ken Massey]] writes that an advantage of computer rating systems is that they can "objectively track all" 351 college basketball teams, while human polls "have limited value".<ref>{{cite tweet |author= Ken Massey |author-link= Ken Massey |user= masseyratings|number= 529238966965661696 |date= 3 Nov 2014 |title= @kenpomeroy human polls have limited value. Computer systems can objectively track all the teams. http://www.masseyratings.com/cb/compare.htm #all351 |access-date= 9 Nov 2014 }}</ref> Computer ratings are verifiable and repeatable, and are comprehensive, requiring assessment of all selected criteria. By comparison, rating systems relying on human polls include inherent human subjectivity; this may or may not be an attractive property depending on system needs.

== History ==

Sports ratings systems have been around for almost 80 years, when ratings were calculated on paper rather than by computer, as most are today.  Some older computer systems still in use today include: [[Jeff Sagarin|Jeff Sagarin's]] systems, the ''[[New York Times]]'' system, and the [[Dunkel Index]], which dates back to 1929. Before the advent of the college football playoff, the [[Bowl Championship Series]] championship game participants were determined by a combination of expert polls and computer systems.

== Theory ==

Sports ratings systems use a variety of methods for rating teams, but the most prevalent method is called a power rating.  The power rating of a team is a calculation of the team's strength relative to other teams in the same league or division.  The basic idea is to maximize the number of [[transitive relation]]s in a given data set due to game outcomes.  For example, if A defeats B and B defeats C, then one can safely say that A>B>C.

There are obvious problems with basing a system solely on wins and losses.  For example, if C defeats A, then an [[Intransitivity|intransitive]] relation is established (A&nbsp;>&nbsp;B&nbsp;>&nbsp;C&nbsp;>&nbsp;A) and a ranking violation will occur if this is the only data available.  Scenarios such as this happen fairly regularly in sports—for example, in the [[2005 NCAA Division I-A football season]], [[Pennsylvania State University|Penn State]] beat [[Ohio State University|Ohio State]], Ohio State beat [[University of Michigan|Michigan]], and Michigan beat Penn State.  To address these logical breakdowns, rating systems usually consider other criteria such as the game's score and where the match was held (for example, to assess a [[home field advantage]]).  In most cases though, each team plays a sufficient number of other games during a given season, which lessens the overall effect of such violations.

From an [[academic]] perspective, the use of [[linear algebra]] and [[statistics]] are popular among many of the systems' authors to determine their ratings. Some academic work is published in forums like the [[MIT Sloan Sports Analytics Conference]], others in traditional statistics, mathematics, psychology, and computer science journals.

If sufficient "inter-divisional" league play is not accomplished, teams in an isolated division may be artificially propped up or down in the overall ratings due to a lack of correlation to other teams in the overall league.  This phenomenon is evident in systems that analyze historical college football seasons, such as when the top [[Ivy League]] teams of the 1970s, like [[Dartmouth College|Dartmouth]], were calculated by some rating systems to be comparable with accomplished powerhouse teams of that era such as [[University of Nebraska|Nebraska]], [[University of Southern California|USC]], and [[Ohio State University|Ohio State]].  This conflicts with the subjective opinion that claims that while good in their own right, they were not nearly as good as those top programs.  However, this may be considered a "pro" by non-[[Bowl Championship Series|BCS]] teams in Division I-A college football who point out that ratings systems have proven that their top teams belong in the same strata as the BCS teams.  This is evidenced by the 2004 [[University of Utah|Utah]] team that went undefeated in the regular season and earned a BCS bowl bid due to the bump in their overall BCS ratings via the computer ratings component.  They went on to play and defeat the [[Big East Conference (1979–2013)|Big East Conference]] champion [[University of Pittsburgh|Pittsburgh]] in the 2005 [[Fiesta Bowl]] by a score of 35–7.  A related example occurred during the 2006 [[NCAA Men's Division I Basketball Championship|NCAA men's basketball tournament]] where [[George Mason University|George Mason]] were awarded an at-large tournament bid due to their regular season record and their [[Ratings Percentage Index|RPI rating]] and rode that opportunity all the way to the [[NCAA Men's Division I Basketball Championship|Final Four]].

Goals of some rating systems differ from one another. For example, systems may be crafted to provide a perfect retrodictive analysis of the games played to-date, while others are predictive and give more weight to future trends rather than past results.  This results in the potential for misinterpretation of rating system results by people unfamiliar with these goals; for example, a rating system designed to give accurate [[point spread]] predictions for gamblers might be ill-suited for use in selecting teams most deserving to play in a championship game or tournament.

==Rating considerations==

===Home advantage===
[[File:France national team fans.jpg|thumb|right|France national basketball team fans]]
{{main article|Home advantage}}
When two teams of equal quality play, the team at home tends to win more often. The size of the effect changes based on the era of play, game type, season length, sport, even [[Circadian advantage|number of time zones crossed]]. But across all conditions, "simply playing at home increases the chances of winning."<ref>{{cite journal|url=http://wjh.harvard.edu/~jamieson/JJ_JASP.pdf|title=The Home Field Advantage in Athletics: A Meta-Analysis|journal=Journal of Applied Social Psychology|last=Jamieson|first=Jeremy P.|access-date=11 November 2014|date=2010|volume=40|issue=7|pages=1819–1848|doi=10.1111/j.1559-1816.2010.00641.x}}</ref> A win away from home is therefore seen more favorably than a win at home, because it was more challenging. Home advantage (which, for sports played on a pitch, is almost always called "home field advantage") is also based on the qualities of the individual stadium and crowd; the advantage in the NFL can be more than a 4-point difference from the stadium with the least advantage to the stadium with the most.<ref>{{cite web|url=http://grantland.com/features/bill-barnwell-best-home-field-advantages/|website=[[Grantland]]|last=Barnwell|first=Bill|date=December 20, 2013|access-date=November 11, 2014|title=Safe at Home}}</ref>

===Strength of schedule===
{{main article|Strength of schedule}}
Strength of schedule refers to the quality of a team's opponents. A win against an inferior opponent is usually seen less favorably than a win against a superior opponent. Often teams in the same league, who are compared against each other for championship or playoff consideration, have not played the same opponents. Therefore, judging their relative win–loss records is complicated.
{{blockquote|text=We looked beyond the record. The committee placed significant value on Oregon's quality of wins.|sign=[[College football playoff]] committee chairman [[Jeff Long (athletic director)|Jeff Long]]|source=press conference, week 12 of the 2014 season,<ref>{{cite news |last=Russo |first=Ralph D. |title=Oregon up to 2 in playoff rankings; TCU to 4th |url=http://hosted2.ap.org/APDEFAULT/347875155d53465d95cec892aeb06419/Article_2014-11-11-FBC-Playoff-Rankings/id-c0570dfb5a0044a798224fbef5da73b8 |agency=Associated Press |date=11 November 2014 |access-date=12 November 2014}}</ref> after ranking 9–1 Oregon above 9–0 Florida State}}
The college football playoff committee uses a limited strength-of-schedule algorithm that only considers opponents' records and opponents' opponents' records<ref>{{cite tweet |author= Stewart Mandel |author-link= Stewart Mandel |user= slmandel|number= 532624545300369408 |date= 12 Nov 2014 |title= Committee doesn't use an SOS ranking. It looks at opponents' record and opponents' opponents record. |access-date= 12 Nov 2014 }}</ref> (much like [[Ratings Percentage Index|RPI]]).

===Points versus wins===
A key dichotomy among sports rating systems lies in the representation of game outcomes. Some systems store final scores as [[Three-valued logic|ternary]] discrete events: wins, draws, and losses. Other systems record the exact final game score, then judge teams based on [[margin of victory]]. Rating teams based on margin of victory is often criticized as creating an incentive for coaches to run up the score, an "unsportsmanlike" outcome.<ref>{{cite news|url=http://www.foxsports.com/collegefootball/story/RICHARDS%253A-BCS-removes-margin-of-victory-element |newspaper=[[Fox Sports]] |date=2001 |access-date=12 November 2014|last=Richards|first=Darryl|title=BCS removes margin-of-victory element}}</ref>

Still other systems choose a middle ground, reducing the [[marginal value]] of additional points as the margin of victory increases. Sagarin chose to clamp the margin of victory to a predetermined number.<ref>{{cite web|url=https://www.usatoday.com/sports/ncaaf/sagarin/|title=NCAAF Jeff Sagarin Ratings|last=Sagarin|first=Jeff|author-link=Jeff Sagarin|date=Fall 2014|access-date=12 November 2014|website=[[USA Today]]}}</ref> Other approaches include the use of a decay function, such as a [[logarithm]] or placement on a [[cumulative distribution function]].

===In-game information===
Beyond points or wins, some system designers choose to include more granular information about the game. Examples include time of possession of the ball, individual statistics, and lead changes. Data about weather, injuries, or "throw-away" games near season's end may affect game outcomes but are difficult to model. "Throw-away games" are games where teams have already earned playoff slots and have secured their playoff seeding before the end of the regular season, and want to rest/protect their starting players by benching them for remaining regular season games. This usually results in unpredictable outcomes and may skew the outcome of rating systems.

===Team composition===
Teams often shift their composition between and within games, and players routinely get injured. Rating a team is often about rating a specific collection of players. Some systems assume [[parity (sports)|parity]] among all members of the league, such as each team being built from an equitable pool of players via a [[Draft (sports)|draft]] or [[free agent|free agency]] system as is done in many major league sports such as the [[National Football League|NFL]], [[Major League Baseball|MLB]], [[National Basketball Association|NBA]], and [[National Hockey League|NHL]]. This is certainly not the case in collegiate leagues such as Division I-A football or men's and women's basketball.

===Cold start===
At the beginning of a season, there have been no games from which to judge teams' relative quality. Solutions to the cold start problem often include some measure of the previous season, perhaps weighted by what percent of the team is returning for the new season. [[ARGH Power Ratings]] is an example of a system that uses multiple previous years plus a percentage weight of returning players.

==Rating methods==
{{inc-sport|date=October 2021}}

===Permutation of standings===
Several methods offer some permutation of traditional standings. This search for the "real" win–loss record often involves using other data, such as point differential or identity of opponents, to alter a team's record in a way that is easily understandable. Sportswriter [[Gregg Easterbrook]] created a measure of Authentic Games, which only considers games played against opponents deemed to be of sufficiently high quality.<ref>{{cite web|url=https://www.espn.com/nfl/story/_/page/TMQWeekEleven141118/more-defensive-penalties-causes-change-nfl-tuesday-morning-quarterback|title=More flags on D spins scoreboards|last=Easterbrook|first=Gregg|author-link=Gregg Easterbrook|date=18 November 2014 |access-date=19 November 2014|website=[[ESPN]]}}</ref> The consensus is that all wins are not created equal.
{{blockquote|text=I went through the first few weeks of games and redid everyone’s records, tagging each game as either a legitimate win or loss, an ass-kicking win or loss, or an either/or game. And if anything else happened in that game with gambling repercussions – a comeback win, a blown lead, major dysfunction, whatever — I tagged that, too.|sign=[[Bill Simmons]], sportswriter|source=[[Grantland]]<ref>{{cite web|url=http://grantland.com/the-triangle/week-8-picks-a-gambling-epiphany/|title=Week 8 Picks: A Gambling Epiphany|last=Simmons|first=Bill|author-link=Bill Simmons|date=24 October 2014 |access-date=19 November 2014|website=[[Grantland]]}}</ref>}}

====Pythagorean====
{{main article|Pythagorean expectation}}
Pythagorean expectation, or Pythagorean projection, calculates a percentage based on the number of points a team has scored and allowed. Typically the formula involves the number of points scored, raised to some exponent, placed in the numerator. Then the number of points the team allowed, raised to the same exponent, is placed in the denominator and added to the value in the numerator. [[Football Outsiders]] has used<ref>{{cite book|last1=Schatz|first1=Aaron|last2=Alamar|first2=Ben|last3=Barnwell|first3=Bill |author4=Bill Connelly |author5=Doug Farrar|title=Football Outsiders Almanac 2011: The Essential Guide to the 2011 NFL and College Football Seasons|url=https://books.google.com/books?id=E84YYAAACAAJ|year=2011|publisher=CreateSpace|isbn=978-1-4662-4613-3|page=xviii}}</ref>
:<math>\text{Pythagorean wins} = \frac{\text{Points For}^{2.37}}{\text{Points For}^{2.37} + \text{Points Against}^{2.37}}\times \text{Games Played}.</math>
The resulting percentage is often compared to a team's true winning percentage, and a team is said to have "overachieved" or "underachieved" compared to the Pythagorean expectation. For example, [[Bill Barnwell]] calculated that before week 9 of the 2014 NFL season, the [[Arizona Cardinals]] had a Pythagorean record two wins lower than their real record.<ref>{{cite web|url=http://grantland.com/the-triangle/nfl-at-the-half-breaking-down-the-numbers/|title=NFL at the Half: Breaking Down the Numbers|last=Barnwell|first=Bill|author-link=Bill Barnwell|date=November 5, 2014|access-date=January 7, 2015|website=[[Grantland]]}}</ref> [[Bill Simmons]] cites Barnwell's work before week 10 of that season and adds that "any numbers nerd is waving a “REGRESSION!!!!!” flag right now."<ref>{{cite web|url=http://grantland.com/the-triangle/revisiting-the-y2k-compliant-quarterbacks/|last=Simmons|first=Bill|author-link=Bill Simmons|date=7 November 2014|access-date=10 November 2014|title=Revisiting the Y2K-Compliant Quarterbacks}}</ref> In this example, the Arizona Cardinals' regular season record was 8-1 going into the 10th week of the 2014 season. The Pythagorean win formula implied a winning percentage of 57.5%, based on 208 points scored and 183 points allowed. Multiplied by 9 games played, the Cardinals' Pythagorean expectation was 5.2 wins and 3.8 losses. The team had "overachieved" at that time by 2.8 wins, derived from their actual 8 wins less the expected 5.2 wins, an increase of 0.8 overachieved wins from just a week prior.

===Trading "skill points"===
{{main article|Elo rating system}}
Originally designed by [[Arpad Elo]] as a method for ranking chess players, several people have adapted the Elo rating system for team sports such as basketball, soccer and American football. For instance, [[Jeff Sagarin]] and [[FiveThirtyEight]] publish NFL football rankings using Elo methods.<ref>{{cite web|url=https://fivethirtyeight.com/features/introducing-nfl-elo-ratings/|title=Introducing NFL Elo Ratings|last=Silver|first=Nate|author-link=Nate Silver|date=4 September 2014|access-date=10 November 2014|website=[[FiveThirtyEight]]}}</ref> Elo ratings initially assign strength values to each team, and teams trade points based on the outcome of each game.

===Solving equations===

Researchers like Matt Mills use [[Markov chains]] to model college football games, with team strength scores as outcomes.<ref>{{cite web|url=http://thespread.us/continuous-markov-ratings.html|last=Mills|first=Matt|website=The Spread|date=21 December 2014|access-date=21 December 2014|title=Using Continuous-Time Markov Chains to Rank College Football Teams}}</ref> Algorithms like Google's [[PageRank]] have also been adapted to rank football teams.<ref>{{cite web|url=https://www.linkedin.com/pulse/ranking-nfl-teams-using-network-science-konstantinos-pelechrinis?trk=prof-post/|website=[[LinkedIN]]|date=17 March 2016|access-date=17 March 2016|title=Ranking NFL teams using Network Science}}</ref><ref>{{cite web|url=https://www.reddit.com/r/CFBAnalysis/comments/2ppfg3/modifying_googles_page_ranking_algorithm_to_rank/|website=[[Reddit]]|date=21 December 2014|access-date=22 December 2014|title=Modifying Google's Page Ranking Algorithm to rank teams}}</ref>

==List of sports rating systems==
* [[Advanced NFL Stats]], United States of America [[National Football League]]
* [[ARGH Power Ratings]]
* [[ATP rankings]], international [[tennis]]
* [[Colley Matrix]]
* [[Dickinson System]], United States of America [[college football]]
* [[Pomeroy College Basketball Ratings]], United States of America [[college basketball]]
* [[Ratings Percentage Index|Ratings Percentage Index (RPI)]], United States of America [[National Collegiate Athletic Association|NCAA]] [[College basketball|basketball]], [[college baseball|baseball]], [[college softball|softball]], [[college hockey|hockey]], [[College soccer in the United States|soccer]], [[college lacrosse|lacrosse]], and [[volleyball]]
* [[Smithman Qualitative Index]], United States of America [[soccer]] - obsolete
* [[TrueSkill]], a Bayesian ranking system inspired by the [[Glicko rating system]]<ref>{{cite journal | url = http://jmlr.csail.mit.edu/papers/volume12/weng11a/weng11a.pdf | title = A Bayesian Approximation Method for Online Ranking | journal = Journal of Machine Learning Research | volume = 12 | date = 2011 | pages =  267–300 | last1 = Weng | first1 = Ruby C. | last2 = Lin | first2 = Chih-Jen }}</ref>

===Bowl Championship Series computer rating systems===
In collegiate American football, the following people's systems were used to choose teams to play in the national championship game.
* [[Anderson & Hester]] / ''Seattle Times''
* [[Billingsley Report|Richard Billingsley]]
* [[Colley Matrix|Wes Colley]] / ''Atlanta Journal-Constitution''
* [[Dunkel System|Richard Dunkel]]
* [[Massey College Football Ratings|Kenneth Massey]]
* [[Matthews Grid Ratings|Herman Matthews]] / Scripps Howard
* ''New York Times''
* [[David Rothman (statistician)|David Rothman]]
* [[Jeff Sagarin]] / ''USA Today''
* [[Peter Wolfe (sports rankings)|Peter Wolfe]]

==Further reading==

===Bibliographies===
* {{cite web|url=http://homepages.cae.wisc.edu/~dwilson/rsfc/rate/biblio.html|last=Wilson|first=David|title=Bibliography on College Football Ranking Systems|website=[[University of Wisconsin–Madison]] |access-date=18 November 2014}}

===Popular press===
* {{cite web|url=http://thepowerrank.com/2014/11/24/how-to-understand-college-football-analytics-the-ultimate-guide/|last=Feng|first=Ed|website=The Power Rank|date=24 November 2014|title=How to understand college football analytics – the ultimate guide}}
* {{cite news|url=https://www.nytimes.com/2012/10/24/sports/ncaafootball/college-football-computer-rankings-proliferate.html?_r=0|newspaper=[[New York Times]]|last=Mather|first=Victor|date=October 23, 2012|title=College Football Rankers by the Dozen Ask the No. 1 Question}}
* Wayne Winston is a professor of decision sciences at [[Indiana University]] and was a classmate of Jeff Sagarin at MIT.<ref>{{cite web|url=http://www.kelley.iu.edu/ODT/NewsEvents/News/NewsReleases/page42205.html|title=Wayne Winston: Analytics in the World of Sports|website=Indiana University Bloomington - Kelley School of Business - Operations & Decisions Technologies|date=Nov 25, 2013 |access-date=8 Nov 2014}}</ref> He published several editions of a text on the Microsoft Excel spreadsheet software that includes material on ranking sports teams, as well as a book focused directly on this topic. He and Sagarin created rating systems together.<ref>{{cite news|url=http://www.washingtontimes.com/news/2004/apr/13/20040413-121657-1462r/?page=all|title=Numbers game|newspaper=[[Washington Times]]|date=April 13, 2004|access-date=8 Nov 2014}}</ref>
** {{cite book|last=Winston|first=Wayne L.|title=Mathletics: How Gamblers, Managers, and Sports Enthusiasts Use Mathematics in Baseball, Basketball, and Football|url=https://books.google.com/books?id=jTsJcTdzOGAC&pg=PR11|year=2012|publisher=Princeton University Press|isbn=978-1-4008-4207-0}}
** {{cite book|last=Winston|first=Wayne L.|title=Microsoft® Excel Data Analysis and Business Modeling|url=https://books.google.com/books?id=h2T403pH_GMC|year=2009|publisher=Microsoft Press|isbn=978-0-7356-3714-6}}

===Academic work===
* {{cite journal|title=Ranking rankings: an empirical comparison of the predictive power of sports ranking methods|journal=Journal of Quantitative Analysis in Sports|first1=Daniel |last1=Barrow |first2= Ian |last2=Drayer |first3=Peter |last3=Elliott |first4=Garren |last4=Gaut |first5=Braxton |last5=Osting|volume=9|issue=2|date=May 2013|doi= 10.1515/jqas-2013-0013 |s2cid=199665454 |issn=1559-0410}}
** Much of this information is available at {{PDFwayback|date=20141029012933|url=http://www.math.duke.edu/~idrayer/2012REU.pdf|title=Sports Rankings REU Final Report 2012: An Analysis of Pairwise-Comparison Based Sports Ranking Methods and a Novel Agent-Based Markovian Basketball Simulation}}
* {{cite journal|title=Comparing Team Selection and Seeding for the 2011 NCAA men's basketball tournament|journal=Journal of Quantitative Analysis in Sports|first1=Kathy L. |last1=Gray |first2= Neil C. |last2= Schwertman |volume=8|issue=1|date=March 2012|doi= 10.1515/1559-0410.1369 |s2cid=121322446 |issn=1559-0410}}
* {{cite web|last=Massey|first=Ken|author-link=Kenneth Massey|date=Spring 1997|url=https://www.masseyratings.com/theory/massey97.pdf|title=Honors Project in Mathematics}} available at {{PDFwayback|date=20141029025444|url=http://74.207.231.132/theory/massey97.pdf|title=Statistical Models Applied to the Rating of Sports Teams}}
* {{cite journal|url=http://www.davemease.com/papers/football.pdf|last=Mease|first=David|title=A Penalized Maximum Likelihood Approach for the Ranking of College Football Teams Independent of Victory Margins|doi=10.1198/0003130032396|journal=The American Statistician|volume=57|issue=4|date=2003|pages=241–248|s2cid=2372150 }}

== References ==
{{Reflist|30em}}

{{Sports rating systems}}
{{Sport world rankings}}
{{Sport}}

[[Category:Sports records and statistics]]
[[Category:Sports terminology]]
[[Category:Sports rankings|*]]
[[Category:Sports science]]
[[Category:Sports business|Rating system]]
[[Category:Baseball statistics]]
[[Category:Basketball statistics]]
[[Category:Analytics]]
[[Category:American football records and statistics]]