Autonomous Combat Achieved, LLM Outside Critical Path
On August 9, 2026, an AI agent played an entire solo combat of Slay the Spire 2 without any human intervention. This session involved 7 turns and 35 distinct actions. While this represents a single recorded session and not a rigorously audited benchmark, it marks a significant step in autonomous gameplay for complex strategy titles. Crucially, none of the 35 actions taken by the agent were directly decided by a large language model (LLM). Instead, each play was determined by a local, score-based greedy policy operating within a Python bridge. The LLM, named Hermes, managed the system but remained outside the critical decision-making loop for in-game actions. The project's most challenging hurdles were overcome not by advancing AI sophistication, but by the game itself providing clear negative feedback – by saying 'no' – and the agent's subsequent ability to learn from these rejections.
The 'No' as a Learning Signal
The core insight driving this project's success lies in how the agent learned to navigate the game's complex decision space. Unlike systems that aim for a perfect predictive model of optimal play from the outset, this agent's breakthrough came from processing explicit denials from the game. When an action was invalid or impossible, the game's feedback was a simple, unambiguous 'no.' This signal, rather than complex probabilistic reasoning, became the primary driver for improving the agent's strategy. The agent was not trying to guess the best move; it was learning what moves were definitively *not* the best, or not possible, based on the game's direct response. This approach is akin to a human learning a new game by trial and error, where hitting a wall and understanding *why* you hit it is more instructive than simply being told the correct path.
Greedy Policy and LLM Orchestration
The decision-making architecture employed a score-based greedy policy executed locally. This means that at each step, the agent evaluated available actions based on a predefined scoring mechanism and selected the highest-scoring option. This policy is computationally efficient and deterministic, making it suitable for real-time gameplay. The LLM, Hermes, served as the orchestrator, managing the overall state of the game, interpreting game events, and feeding information to the greedy policy. However, Hermes did not directly choose which card to play or which enemy to target. Its role was more supervisory, ensuring the game environment was correctly represented and that the greedy policy had the necessary inputs. This separation of concerns allowed the LLM to focus on higher-level strategy and state management, while the direct action selection was handled by a simpler, more immediate decision-making process. This architecture proved more effective than attempting to imbue the LLM with the capacity for every micro-decision, especially when those decisions could be learned through simple feedback loops.
The Two Hardest Problems Solved by Rejection
The project encountered two significant challenges that were unexpectedly resolved by the game's 'no' responses. The first was managing the sheer combinatorial explosion of possible actions and their consequences in a game like Slay the Spire 2. With numerous cards, relics, and enemy states, predicting the optimal move in all scenarios is computationally prohibitive for even advanced models. The agent's greedy policy, coupled with learning from invalid moves, effectively pruned the decision tree. Instead of exploring every branch, it learned to avoid branches that led to invalid states. The second challenge was handling the game's nuanced rules and interactions, which can be difficult to capture exhaustively in a training dataset or a set of explicit rules. When the game rejected an action, it implicitly communicated a rule violation or an impossible state. By logging these rejections and associating them with the preceding state, the agent built a more robust understanding of the game's mechanics than a purely predictive model might achieve. This iterative process of attempting, being rejected, and adjusting is a powerful, albeit simple, form of reinforcement learning.
Implications for Autonomous Agents in Complex Games
This approach has broad implications for developing autonomous agents in other complex, strategic games. It suggests that systems do not always require sophisticated, human-like reasoning to achieve high levels of performance. By leveraging the game's inherent feedback mechanisms – its explicit 'yes' and 'no' – agents can learn to navigate complex state spaces more efficiently. This method is particularly valuable in games where the rules are intricate, emergent, or difficult to fully codify. The reliance on a simpler, local decision-making policy augmented by explicit negative feedback can also lead to more interpretable and debuggable AI systems. Developers can more easily understand why an agent made a particular choice by examining the scoring function and the specific 'no' signals that guided its learning. This contrasts with black-box LLM approaches where the reasoning process can be opaque. The success of this agent in Slay the Spire 2 demonstrates that sometimes, the most effective AI development path is not about making the AI smarter, but about making it better at listening to what the system is telling it.
Future Directions and Open Questions
While this achievement is notable, several questions remain. The reliance on a session record, rather than a fully archived log, means that reproducing the exact conditions for further analysis or benchmarking is challenging. Future work will need to focus on robust logging and versioning of game artifacts to ensure reproducibility. Furthermore, exploring the scalability of this 'learning from 'no'' approach to even more complex games with less explicit feedback mechanisms will be critical. What happens when the game doesn't say 'no,' but simply leads the agent down a suboptimal path with no immediate, obvious failure state? How can this method be combined with more traditional reinforcement learning techniques to accelerate learning and improve performance further? The agent's success opens a new avenue for AI development in games, one that prioritizes learning from constraints over pure predictive power.