The central insight behind the work is that rankings, which look deceptively simple on the surface, are statistically very complicated objects. Rose Graves, a doctoral student studying statistics at Rice and the study’s corresponding author, explained the challenge: two experts may rank different numbers of items, disagree about the order, or even change their opinions over time. Traditional approaches to combining rankings, such as simply averaging a player’s position across several lists, can mask meaningful information, especially when a player is omitted from one list altogether or when assessments shift substantially between seasons. An average also says nothing about how confident we should be in the resulting ordering. BMRR takes a fundamentally different approach, treating each published ranking not as a final verdict but as evidence about an underlying, latent value for each player, and then using a hierarchical Bayesian framework to combine that evidence in a principled way.

The Bayesian machinery at the heart of the method is what allows it to handle the messiness of real-world ranking data. Because the model is hierarchical, it can incorporate incomplete rankings and ties, borrow information across years, and estimate how closely each individual source aligns with the overall consensus. Crucially, unlike a single definitive list produced by conventional aggregation, the output includes a measure of uncertainty. The model shows when the data strongly favor one player over another and when two players are effectively too close to separate. That distinction matters because rankings can carry real economic consequences. In professional baseball, perceptions of player value can influence trades, salary negotiations, arbitration hearings, and draft decisions. Marina Vannucci, the Noah Harding Professor of Statistics at Rice and one of the authors of the paper, noted that rather than asking only who ranks first, the framework lets researchers ask how confident they are in that conclusion, why someone may be ranked highly, and how much agreement actually exists among the people doing the ranking. That kind of uncertainty quantification, she emphasized, can be extremely important when rankings are being used to inform decisions.

For their real-world test case, the researchers turned to annual lists of Major League Baseball’s best players, analyzing preseason rankings published by ESPN, CBS, Bleacher Report, Yahoo Sports, and MLB itself from 2021 through 2024. Each outlet ranked its top 100 players, but the lists varied both in who was included and where players were placed. To create a consistent pool for comparison, the researchers focused on 55 players who appeared in at least one outlet’s rankings in each of the four years. This dataset captures precisely the features that make ranking data difficult to analyze: overlapping but non-identical item sets, divergent orderings, and temporal change. By modeling all of this simultaneously, BMRR could estimate a consensus trajectory for each player across the entire period rather than producing four disconnected snapshots.

Beyond producing a consensus, the model allowed the researchers to investigate what characteristics were actually associated with higher rankings. They incorporated three player-level variables into the regression framework: age, base salary, and wins above replacement, or WAR, a widely used statistic that estimates a player’s overall contribution to his team. All three turned out to be associated with rankings. Higher WAR had the strongest positive effect, followed closely in magnitude by age, with younger players tending to rank more favorably. Higher salaries were also associated with higher rankings. Taken together, the findings suggest that media rankings reflect not just recent on-field performance but also expectations about a player’s future potential and value. This ability to attach covariates to the ranking process is one of the features that distinguishes BMRR from earlier rank-aggregation techniques, which typically treat the incoming lists as raw inputs without asking what drives them.

The analysis also produced some recognizable results at the top of the consensus. For the 2023 season, the model’s four highest-ranked players were Shohei Ohtani, Aaron Judge, Mike Trout, and Mookie Betts. All four were selected as All-Star starters that season, and Ohtani went on to win the American League Most Valuable Player Award. But the more striking demonstration of the method’s power came from looking across multiple years, which allowed the researchers to capture something a one-season ranking cannot: the difference between sustained excellence and a rapid rise or fall. Ohtani, for example, showed unusually high variability across the four-year period. After injuries affected his earlier seasons, only one of the five outlets included him in its top 100 for 2021. By 2022, two outlets ranked him No. 1, and by 2024 all five placed him in their top 10. The model captured both his eventual position near the top and the dramatic trajectory that got him there.

The framework can also calculate the probability that one player should rank above another, which makes it potentially useful for head-to-head decisions. Among four infielders who entered free agency after the 2024 season, for example, the analysis showed strong consensus that Alex Bregman, Pete Alonso, and Paul Goldschmidt ranked above Willy Adames. In practical terms, this means the model can translate a tangle of conflicting expert opinions into a quantified statement of the form: the evidence supports this ordering with a given degree of confidence. For decision-makers weighing signings, trades, or contract disputes, that is a meaningfully different kind of output from a bare list of names, because it comes with an explicit statement of how much the underlying sources actually agree.

Notably, BMRR does not only evaluate the players being ranked; it can also reveal how individual rankers behave. In the baseball analysis, MLB’s own rankings aligned most closely with the model’s overall consensus, followed closely by ESPN. Bleacher Report and Yahoo deviated more substantially from the aggregate rankings, and the authors observed that Yahoo’s lists, for instance, appeared more inclined to elevate younger, rising players over established veterans. That does not necessarily make one ranking right and another wrong. Instead, the model provides a quantitative way to see which sources tend to track the broader consensus and which bring a genuinely different perspective. When the researchers tested BMRR on simulated data, it consistently performed well compared with existing rank-aggregation methods, and its advantage was especially pronounced when rankings were incomplete, a common real-world scenario in which different people rank only their preferred subset of a much larger group.

While Major League Baseball provided a compelling proving ground, the researchers designed BMRR for a much broader problem. Rankings appear almost everywhere: search engines order websites, voters rank candidates, experts prioritize intelligence information, and clinicians may rank treatment preferences. Existing statistical approaches often struggle when different raters evaluate different items, when rankings are repeated across multiple criteria or points in time, or when analysts want to incorporate additional information about the items being ranked. Dan Kowal, a former Rice statistics faculty member now at Cornell University and second author on the paper, observed that the biggest challenges when working with ranking data are remarkably similar across different fields and industries, and that the team believes BMRR can be impactful in many areas beyond baseball. The authors have made the code for the method publicly available on GitHub so that other statisticians can apply it to their own data. As Graves put it, wherever multiple sources disagree about what belongs at the top, the framework offers a way to identify a consensus while measuring how confident that consensus should be, turning one of the oldest arguments in sports into a problem that statistics can finally address with rigor.

Subject of Research: Bayesian statistical methods for aggregating conflicting sports rankings

Article Title: Who’s really No. 1? Rice University statisticians build better way to make sense of competing sports rankings

Article References: Who’s really No. 1? Rice University statisticians build better way to make sense of competing sports rankings. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: Bayesian statistics, rank aggregation, sports analytics, Major League Baseball, uncertainty quantification, hierarchical models, player rankings, wins above replacement, Rice University, Cornell University, Journal of Quantitative Analysis in Sports, rank regression