Leela Chess Zero Blog

Leela Chess Zero is the open source successor of Alpha Zero, and has been the second strongest engine (except a few times where it bested stockfish) for the last few years. [1] https://lczero.org/

Official website: http://lczero.org/ GitHub: https://github.com/LeelaChessZero/ Still a very active community project. Didn't see the link posted here in the comments yet, definitely worth checking out. Most of the devs and serious contributors are active in the discord channel.

If you would like to use "AlphaZero lite" to analyze games you can opt for LCZero (Leela Chess Zero) http://lczero.org/ These guys have done a fantastic job on resources that's shoestring budget compared to AlphaZero. Also our own clone BellaChess https://code.aihello.com/aihello/BellaChess/ it's not really "zero" as it started off with supervised learning of human chess games and then moved on to CPU+GPU engine

James put together a really nice summary of the ideas and the projects! It was almost a year ago that lc0 was launched, since then the community (led by Alexander Lyashuk, author of the current engine) has taken it to a totally different level. Follow along at http://lczero.org! Gcp has also done an amazing job with Leela Zero, with a very active community on the Go side. http://zero.sjeng.org Of course, DeepMind really did something amazing with AlphaZero. It’s hard to overstate how dominant minimax search has been in chess. For another approach (MCTS/NN) to even be competitive with 50+ years of research is amazing. And all that without any human knowledge! Still, Stockfish keeps on improving - Stockfish 10 is significantly stronger than the version AlphaZero played in the paper (no fault of DeepMind; SF just improves quickly). We need a public exhibition match to setttle the score, ideally with some GM commentary :). To complete the links you can watch Stockfish improve here: http://tests.stockfishchess.org.

https://lczero.org/blog/2024/02/how-well-do-lc0-networks-compare-to-th… The best neural network chess engine's authors wrote about this deepminds publication.

As described in the OP's blog post https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world… - one of the incredible things here is that the standard GPT architecture, trained from scratch from PGN strings alone, can intuit the rules of the game from those examples, without any notion of the rules of chess or even that it is playing a game. Leela, by contrast, requires a specialized structure of iterative tree searching to generate move recommendations: https://lczero.org/dev/wiki/technical-explanation-of-leela-chess-zero/ Which is not to diminish the work of the Leela team at all! But I find it fascinating that an unmodified GPT architecture can build up internal neural representations that correspond closely to board states, despite not having been designed for that task. As they say, attention may indeed be all you need.

I’d use Leela Chess Zero over Stockfish for an odds game. Leela can output win/draw/loss probability instead of centipawns so you can pick whatever move wins most often instead of the move that best delays losing. They also just implemented a contempt function which the latest stockfish doesn’t support: https://lczero.org/blog/2023/07/the-lc0-v0.30.0-wdl-rescale/contempt-i… They setup a lichess bot to allow you to play Leela at knight odds: https://lczero.org/blog/2023/11/play-with-knight-odds-against-lc0-on-l…

Thank you! This is definitely something we should look into in the future (hopefully with community help); as of now, training infrastructure and model convergence are the highest priorities. That said, we welcome all ideas of ways to motivate more volunteers to join the experiments, because Learning@home team comes from a distributed DL background with limited volunteer computing expertise. Also, I believe that for some projects (e.g. GPT-3 replication effort) people would want to join the network regardless of the incentive mechanism, as demonstrated by Leela Chess Zero [1]. [1] http://lczero.org/

LeelaZero is strictly used for the Go AI nowadays, the chess version changed their name to LC0 (or Leela Chess Zero, or LCZero), and the website is at [0] AlphaZero is not involved in any battle with LC0 or Stockfish, as no one outside of Deepmind has accessed to it. The battle for chess supreme is between LC0 and Stockfish, with both trading blows pretty much every update :-) [0] https://lczero.org/

And it's already been done for the chess version, in a community effort[0]. Well, "recomputing the weights" isn't plausible for a neural network of this size, but computing weights that give the network a similar level of performance. [0] https://lczero.org/

Related to AlphaZero is the Leela Chess Zero project, an independent project people from across the glob contribute CPU/GPU resources to: https://lczero.org/#tab-elo-estimates. One of the main developers on LCZero is also one of the Stockfish developers. It's been having an interesting time in various competitions, and much like the match you point out, it makes some decidedly more "human" moves, little stabs to try and get out of a draw situation that Stockfish won't due to its hard-coded biases. Here's an interesting game from TCEC 14 https://www.youtube.com/watch?v=rhI4DKGSjtk Ultimately it came second to Stockfish, but that was the highest placing it has had so far, and they keep refining and improving it all the time.

http://zero.sjeng.org/ for AlphaZero playing Go, and http://lczero.org/ is at least par with AlphaZero Chess based off match reproductions.

It's a shame they still played against 2016 Stockfish (Stockfish 8), when Stockfish 9 or Stockfish Dev were available (Stockfish 10 is out now, but only very recently, so I can understand why they didn't use it). Their results show that they are only just barely stronger than Stockfish 8, but Stockfish 9 and 10 are stronger than 8 as well. EDIT: Also meant to include a shout-out to http://www.lczero.org/ which is an open source implementation of AlphaZero chess. Here is their forum post for this paper: https://groups.google.com/forum/#!topic/lczero/TfmaNHI99gk SECOND EDIT: I was wrong! They did play against a newer SF than 8, specifically, SF at this commit: https://github.com/official-stockfish/Stockfish/commit/b508f9561cc2302… , which was about 2 weeks before SF 9 was released, so maybe it is close in strength to SF 9.

AlphaZero was playing Stockfish 8. This is done using Stockfish dev which is more than 110 ELO higher than Stockfish 8 (see https://github.com/glinscott/fishtest/wiki/Regression-Tests) and higher than what was reported for AlphaZero. Check out the Leela project, which is trying to reproduce and improve the AlphaZero project to also beat Stockfish dev: http://lczero.org/ (at this point they are about equal with Stockfish 9)

There's the TCEC [0] which is a big thing in some circles. Stockfish does lose every now and then against top engines. [1] Usually it's two different engines playing against one another, though. Like Leela Chess Zero [2] vs. Stockfish. In that hypothetical of running 2 instances of Stockfish against one another on a modern laptop, with the key difference being minutes of compute time, it'd probably be very close to 100% of draws. Depending on how many games you run. So, if you run a million games, there's probably some outliers. If you run a hundred, maybe not. When it comes to actually solved positions, the 7-piece tables take around 1TB of RAM to even run. These tablebases are used by Stockfish when you actually want to run it at peak strength. [3] [0]: https://tcec-chess.com [1]: https://lichess.org/broadcast/tcec-s28-leagues--superfinal/match/KeswE… [2]: https://lczero.org [3]: https://github.com/syzygy1/tb

This is a slight tangent to the article, but in the hope HN readers will find this as fascinating as I have recently... There's a three-way battle developing between AlphaZero (as described in the article, courtesy of DeepMind), LCZero[5] (derived from LeelaZero[1] -- an open source interpretation of the same principles as AZ), and Stockfish[2] (a long-standing open source chess engine that has recently begun including neural network support). The 'Top Chess Engine Championship'[3] seems to be a good way to follow the latest news; they also stream matches live on their website[4] (it is quite an information-dense site). You can play against an up-to-date implementation of Stockfish in your browser -- no registration or signup required -- at https://lichess.org [1] - https://zero.sjeng.org/ [2] - https://stockfishchess.org/ [3] - https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship [4] - https://tcec-chess.com/ [5] - https://www.lczero.org/ Edit: correct LeelaZero -> LCZero

At a minimum: building the opening book alone will require human intervention. When you build an opening book, you need to pick-and-choose which engines will self play. Will you build an opening book vs Stockfish? Komodo? LeelaZero? If so, how will you generate these LeelaZero games? You'll have to build a computer (or rent one from AWS) to play these LeelaZero games. What are the time-controls of matches? Self-play at 40-minutes + 15-second increment means that you'll only create a game every hour or so. Spend 30-days building databases at 40-minute + 15-second increment games, and you'll only reach ~720 games of analysis per month of preparation. Self-play at 1-minutes + 0-second increment results in a game win/loss every 2-minutes (maximum), giving you 21600 generated opening book positions per month of analysis. But these 21600 games are of lower-quality. ---------- Spend 1-month building an anti-Stockfish database, an anti-LeelaZero database, an anti-Komodo database... and you're now 3-months into preparation and there's still Shredder, Johnny, and all other programs that may arrive at the contest. Its not exactly as easy as you think it is. There's no algorithm that automatically builds the best opening book (or "counter" opening book) against your opponents. Its a human choice for what the computer will spend the next-3 months self-analyzing and self-playing. Lets think about how to build anti-LeelaZero seriously. Which of these networks do you download as the…

Yep, exactly. I spent a lot of time trying to figure out better ways for interpreting the evaluations of engines for https://www.schachzeit.com/en/openings/barnes-opening-with-d5/gedult-g… and I ended up liking WDL much better than centipawns. A blunder defined in terms of decreasing your chance of winning by such and such percentage is, to me, a much better definition than a blunder losing such and such material. What does that mean? It makes sense to me now, but it took a long while. Relatedly, there is an interesting thing that lc0 has been doing as well, where it takes the contempt concept even further, and can beat you with queens odd. https://lczero.org/blog/2024/12/the-leela-piece-odds-challenge-what-do… It assumes it is better than you and that it shouldn't just give up because you might be up a knight, rook or even queen.

Yes, the Leela team has worked on a term they call Contempt. (Negative contempt in this case would make the engine seek out less sharp play from whites perspective) In the first link the authour talks about using contempt to seek out/avoid sharp lines. lc0 and nibbler are free, so feel free to try it out if curious. https://github.com/LeelaChessZero/lc0/pull/1791#issuecomment-148674658… https://lczero.org/blog/2023/07/the-lc0-v0.30.0-wdl-rescale/contempt-i…

The lc0 group has switched the result prediction to predict win, loss, and draw probabilities instead of just win/loss. Some information can be found in https://lczero.org/blog/2020/04/wdl-head/

Part of the problem is that it is difficult/impossible to go through the original sources and verify the claims. Another part of the problem is knowing what biases an outlet/reporter has and what parts of a story/the thing they are reporting on is missing out. It's known that a few research papers (e.g. [1], [2]) have been fabricated on various topics. So when you read the news about those, do you read the papers? If you do, can you verify the claims in those papers? It's not possible to do at scale, so you have to trust the reporting on those stories. A lot of reporting on things like that, or releases from major companies, tend to be based on press releases. You also have papers and websites that use others as their source of the information, so any inaccuracies or biases in that initial reporting gets magnified. It also depends on the knowledge of the person reporting -- if they are not an expert in the field they can easily get some of the details wrong. It took a long time, combined effort, and a lot of resources for the Leela Zero team to replicate the results of Alpha Zero [3]. And that's with papers that don't have the source code, or exact details on the training methodology. I've seen quite a lot of papers on things like arXiv that amount to a summary of "we did a thing, here's a table of results" -- this makes it hard to replicate the results. And a lot of papers build on work of others, so quite often you'll need to read and understand hundreds of…

from https://lczero.org/dev/wiki/technical-explanation-of-leela-chess-zero/… Leela uses PUCT (Predictor + Upper Confidence Bound tree search). We evaluate new nodes by doing a playout: start from the root node (the current position), pick a move to explore, and repeat down the tree until we reach a game position that has not been examined yet (or a position that ends the game, called a terminal node). We expand the tree with that new position (assuming non-terminal node) and use the neural network to create a first estimate of the value for the position as well as the policy for continuing moves. In Leela, a policy for a node is a list of moves and a probability for each move. The probability specifies the odds that an automatic player that executes the policy will make that move. After this node is added to the tree, backup that new value to all nodes visited during this playout. This slowly improves the value estimation of different paths through the game tree. When a move is actually played on the board, the chosen move is made the new root of the tree. The old root and the other children of that root node are erased. This is the same search specified by the AGZ paper, PUCT (Predictor + Upper Confidence Bound tree search). Many people call this MCTS (Monte-Carlo Tree Search), because it is very similar to the search algorithm the Go programs started using in 2006. But the PUCT used in AGZ and Lc0 replaces rollouts (sampling playouts to a terminal game…