Building a Chess Coach — Anant Dole and Asbjorn Steinskog, Take Take Take

From Magnus Carlsen to a chess coach in your pocket

Anant opens with the hook: yes, that Magnus Carlsen — widely considered the best player in the world — founded Play Magnus, where both speakers work. The app lets people play, post games, and now get an AI-powered game review that explains moments like a knight capture on e5 leading to mate, complete with “brilliant” labels and commentary about threats, tactics, and plans.

A quick history lesson: chess and AI have been tangled together for decades

Asbjørn runs through the lineage from Claude Shannon’s 1949 paper to Deep Blue beating Kasparov in 1997, then to DeepMind’s AlphaGo and AlphaZero. He frames it as the old split between brute-force “type A” engines and more intuitive “type B” systems — and notes that modern LLMs reintroduced the dream of explanation, even if they still can’t actually play solid chess.

Why LLMs fall apart over the board

The funniest proof is Magnus commenting on an LLM chess tournament from their Oslo office, dunking on Grok for drifting into a Poison Pawn line and then losing badly. Asbjørn’s point is simple: LLMs are trained on language, not board search, so they hallucinate moves — though transformer architectures themselves aren’t the issue, since DeepMind has shown a transformer trained on position evaluations can reach grandmaster strength.

The real pipeline: Stockfish thinks, detectors inspect, the LLM translates

Their solution is to keep the model tightly grounded. First they analyze the whole game with Stockfish, then extract context using detectors for forks, pins, skewers, doubled pawns, and other tactical or structural themes, and they also use Maya from the University of Toronto to estimate what moves humans at, say, 1500 rating would actually find.

Making commentary feel useful instead of generic

That extra context lets them explain not just that a move was bad, but why. In Asbjørn’s example, an opponent’s ...f5 is more than a “bad move” indicator: the system can say it threatens to trap the queen with Bg5, but also explain that capturing in the center defends the escape square and gets the queen out.

The downvote-to-PR loop is the most modern part of the whole thing

If a user reports commentary as bad in the app, it posts to Slack and gets injected into a running Cloud Code channel, which kicks off a commentary-triage skill. The agent can inspect the position, rerun generation, modify prompts or detectors, verify the fix, and ping a human in Slack — so Asbjørn can literally approve and merge from a bus ride if it looks right.

Fast enough to feel instant, even if smarter models exist

Because users want review immediately after a game, they optimized for speed over elaborate reasoning. Their target was sub-3 seconds, and Gemini 3 Flash hit roughly 1 second time-to-first-token and about 3 seconds end-to-end, while slower reasoning models may be fine later for a more patient “chat with your coach” experience.

Their closing lesson: build the plumbing, then let the model speak

They evaluate with 16 chess scenarios pulled from real games, covering tactical patterns, blunders, and hallucination resistance, and use OpenRouter to swap models quickly. The bigger takeaway is broader than chess: separate the data pipeline from language generation, invest in context extraction even if it starts as a painful giant JSON blob, and use autonomous agents plus domain experts to tighten the loop.