I guess Magnus wasn’t worried about his place in my XPAR rankings today. It ranks classical players by move-by-move performance compared with human GM expectation, with a rating aligned to mid-2000s era FIDE Elo.
He ranks 14th! Magnus (and Hikaru, 58th) are already penalized because these rankings reward activity. Only games from the last two years count, and players are ranked according to the lower bound of a 90% confidence interval. Here are updated top 5 lists; open and women’s top 20s below. [1/4]
On the women’s side Anna Muzychuk has the highest rating, and is the only one to crack the top 200 of the open list. It’s close for number 2! Goryachkina has the second-highest XPAR, but lands in fourth, due to a lower-confidence rating having played fewer games than Bibisara and Zhu. [3/4]
1
1
365
This is mostly just for my own amusement :) but I do think it’s working better than I expected.
You can dig into rankings yourself at chessds.com/rankings and read about how they are produced at chessds.com/blog/xpar-a-new-… [4/4]
3
3
286
Here's an explanation of how XPAR chess rankings are produced, the data that supports them, and caveats. The first weekly update is coming soon.
chessds.com/blog/xpar-a-new-…
Elo ratings have worked for chess for decades, but there are new ways to solve the problem now! Using the ML models that power ChessDS I’ve launched XPAR rankings, built from performance measurement that compares play against human GM expectation.
This has been a blast to play with. Check them out yourself at chessds.com/rankings
More details below. This is the History → By Player view, with yearly XPAR for recent World Champions. [1/4]
I’ll be producing this going forward, hopefully weekly matching @MarkTWIC's releases. Thanks Mark! A full write-up soon, but here’s a brief outline of the method, which never reads the actual game result:
- Move-by-move expectation is built by integrating over all moves predicted to have >=1% chance of being played. Each move contributes its change in expectation, given by the result prediction model, weighted by its chance to be played. This is how XPA has always worked, but models are better now.
- Aggregates (at game-, event-, or player-level) are adjusted for opponent strength. This is a small adjustment built from the opponent’s XPA only (never Elo rating) on previous games, and weights recent games, and adult-age games, more heavily.
- The final XPA average is converted to an XPAR rating based on a translation to Elo fit from 2004-2007, before recent deflation took off.
- The rankings apply the recency weighting I mentioned.
At game-level, XPAR has a lot of variance. But aggregated across games it shows a strong correlation with Elo. In fact, it is even more predictive of subsequent tournament results, once players have a few dozen measured games. [3/4]
1
1
101
There’s a ton you can see on the page. Dive into the current rankings, going deep on any player to see all the contributing games and how their ranking has changed. Or look at how the measure performs historically, viewing XPAR over an any selected interval of years, and producing a line chart with up to 20 players over time. Let me know if you spot something interesting! [4/4]
1
57
Finally! ChessDS is live with new state-of-the-art prediction models, covering an expanded corpus of 660K+ fully analyzed games, incorporating improvements from new neural DSC models, and measuring performance with a new Elo-style "XPAR" rating.
It’s been an adventure, with lots of challenges translating improved prediction to improved performance measurement (more below). But the wins are now clear. First is move prediction, which you can find at every ply of every game on site. Accuracy is much improved.
I also compared with the newly released Toby 1.0. On the GM classical set, ChessDS is slightly ahead, but it’s no blowout. This is great work by @IglesiasYosha, which you can download and use on yoru own games to get a very similar readout to ChessDS. [1/3]
Game-level result prediction improved a bit, but gains are more modest. You can switch between the rating-aware and rating-unaware predictions with a toggle on-site. Predictions that don't use rating can give a better sense of the truth of the position, presuming both players are average GMs. These rating-aware predictions are the most accurate you can find anywhere, and are shown at every ply of every game on site. [2/3]
1
1
214
This unblocks the real fun – performance measurement. I’ve been wanting to land better models, and turn XPA into an Elo-style “XPAR” rating. Now it’s done.
Improving accuracy here was surprisingly hard, and in the end I had to leave out the neural DSC models completely when estimating player performance. Even when set to a defined rating, and given no history of the game, these complex models learn to read the strength of the player from the board itself. This corrupts the move-by-move signal XPA is built from.
The new XPAR baseline model uses a worse move predictor, but is a better baseline to measure strength against. Here I show the improved correlation between FIDE rating and XPAR. Given natural variation and Elo error, this is approaching what I believe is theoretical peak performance, and is good enough to power player rankings. More on this, very soon!
Check out the new corpus and XPAR ratings now at chessds.com [3/3]
Today for the first time I’m releasing some of the trained models I built for ChessDS. These are brand-new, Maia-3-style chessformer neural models, independently reimplemented and extended, with a focus on classical GM chess. They easily outperform Maia-3 predicting moves in this population.
I named them “DSC” for ChessDS Chessformer. Play with them, build on them, modify the weights, go nuts! They are freely available, links below, including UCI engines. [1/5]
For ChessDS I’m now working to flow these through the full move and result prediction models, which combine DSC outputs with engine evals and other signals, producing what I believe are the world’s most accurate predictions of GM chess.
The site uses a fleet of models which are retraining now, and then I’ll be re-scoring the full corpus. I’m hopeful I’ll be able to release some of these models too, although it’s more complex. (and honestly I’m anxious to get back to analysis) [4/5]
1
79
Links here: free to use under GPL-3.0-or-later (code) and CC BY 4.0 (weights). Have fun! A public thanks and a backlink to chessds.com works great. Cheers.
- UCI engines: chessds.com/downloads
- GitHub (code, documentation): github.com/ChessDS/chessds-d…
- Hugging Face (model weights): huggingface.co/ChessDS
1
78
The frontier of GM move prediction is advancing.
Building on the awesome work of the Maia team, I’ve been training new models specifically for predicting moves in GM-level chess. I’m calling them “DSC” for DS Chessformer. I’m super excited with these results, and working now to make these models freely available. Should happen this week!
Here’s performance predicting the move played, on my holdout set of 81,939 GM-level games since 1994. These models are designed to be drop-in replacements for Maia-3. They perform substantially better on GM games at comparable model sizes.
While Maia-3 is available in 23M and 79M parameter versions, I’ll also be releasing a 150M parameter model with even better performance. And in fact I’m training a 500M parameter model now too, just because I’m a chess modeling sicko : )
These results are in progress – the 79M and 150M will improve further as I’m extending their training. If you’re using Maia-3 and want to try these out now, let me know!
To be clear this isn't really frontier accuracy -- just the frontier for the neural policy network. For the site I will use this as an input to a full move-prediction model that will also use engine evals and other features to get better performance.
Here are some very preliminary numbers on the improvements there. But this model also takes days to train and this is just from a small test. Real results will be better.
I'm hoping to use the mammoth 500M model for a new version of XPA with high-enough accuracy to support player rankings.
101
Rating-aware predictions in action for today’s game between Rose Atwell and Alice Lee
chessds.com/live/us_women_s_…
4
436















