Llmchess

Llmchess is an AI-powered platform designed to explore and evaluate chess-playing capabilities of large language models through a standardized, web-based interface. Its primary purpose is to provide a controlled environment where different LLMs can be tested, compared, and analyzed as chess agents, using a common protocol and consistent conditions. By focusing on reproducible experiments, Llmchess helps researchers and practitioners better understand how language models reason about and play chess positions and full games.

The platform supports interactions with multiple models via a unified API, enabling side-by-side comparisons of move quality, consistency, and strategic depth. It includes a browser-based chessboard interface for visualizing positions, stepping through moves, and inspecting model outputs in real time. Llmchess also incorporates mechanisms for enforcing legal moves, tracking game state, and preventing common LLM failure modes such as illegal moves or inconsistent board updates. In addition, it is designed to be extensible, allowing integration of new models, custom prompts, and different evaluation setups without major code changes.