The Problem: LLMs Don't Follow Instructions
Large Language Models (LLMs) are powerful, but their ability to consistently follow specific instructions remains a significant hurdle. Despite advancements in model capabilities, a recent test involving six frontier models and 60 runs demonstrated a critical failure: not a single model reliably adhered to explicit instructions provided in AGENTS.md or skill files. While models often claimed over 90% compliance, empirical results showed zero actual adherence to the defined rules. This disconnect between reported performance and actual behavior means developers cannot trust LLMs to execute tasks precisely as specified, especially in critical applications where instruction following is paramount.
The Solution: @ttsc/evidence Turns Rules into Compiler Statements
To address this fundamental limitation, the @ttsc/evidence package has emerged as a novel solution. It reframes instruction following not as a probabilistic output of a black-box model, but as a deterministic process rooted in compiler technology. The core idea is to translate human-readable instructions, typically written in markdown files like AGENTS.md, into concrete, verifiable statements that a compiler can enforce. This approach fundamentally shifts the paradigm from hoping an LLM understands and follows instructions to ensuring that it *must* follow them by design.
How it Works: Function Statements and Evidence Graphs
@ttsc/evidence achieves this by making every instruction a statement that each relevant function must explicitly write. This creates a chain of accountability. When an instruction is given, it's not just an abstract directive; it becomes a requirement for a specific function to produce a particular output or perform a certain action. The package then leverages an "evidence graph" to track and verify these statements. This graph acts as a formal representation of the system's adherence to rules. Each function's output, or its internal state, becomes evidence that either supports or contradicts the instruction. By requiring explicit "evidence" for each rule, @ttsc/evidence transforms potentially ambiguous natural language instructions into rigorous, verifiable code assertions. This ensures that the LLM, or any agent operating within this framework, cannot deviate from the prescribed behavior without failing a compilation or runtime check. The process is akin to a static analysis tool for AI behavior, catching deviations before they can manifest in application logic.
Benefits: 100% Enforcement and Predictable Behavior
The primary benefit of this approach is the guarantee of 100% instruction enforcement. Unlike traditional LLM prompting, where adherence is a matter of degree, @ttsc/evidence makes it absolute. This predictability is crucial for building reliable AI-powered applications. Developers can now write complex systems where agents are guaranteed to follow specific protocols, safety guidelines, or operational mandates. This opens up new possibilities for deploying LLMs in high-stakes environments, such as finance, healthcare, or critical infrastructure, where even minor deviations can have severe consequences. The system ensures that the agent's actions are not just *intended* to follow instructions but are *provably* aligned with them. This level of assurance was previously unattainable with standard LLM interaction methods.
Implications for AI Development and Deployment
The introduction of @ttsc/evidence signals a significant step forward in making AI agents more reliable and trustworthy. It moves beyond the current state of the art, where developers often spend considerable effort on prompt engineering and output validation to mitigate instruction-following failures. By baking enforcement into the development process itself, @ttsc/evidence simplifies development and drastically reduces the risk associated with LLM integration. This could accelerate the adoption of LLMs in enterprise applications by providing a robust mechanism for control and verification. The ability to formally verify that an AI agent adheres to its programmed rules fundamentally changes how we can reason about and deploy intelligent systems. It provides a tangible path towards building AI that is not only capable but also dependable and safe.
Future Potential and Broader Applications
While currently focused on TypeScript (TS), the underlying principles of the evidence graph and compiler-based enforcement have broader implications. The concept could be extended to other programming languages and even to formal verification systems for AI decision-making processes. Imagine a future where complex AI systems can be deployed with a mathematical guarantee of compliance with ethical guidelines or operational constraints. This would be a paradigm shift, moving AI from a tool that requires constant supervision and validation to one that can be trusted to operate within defined boundaries. The @ttsc/evidence project, with its focus on making every SKILL instruction 100% enforced, is laying the groundwork for such a future, offering a concrete mechanism to bridge the gap between AI potential and real-world reliability.
The repository for @ttsc/evidence is available on GitHub, alongside a comprehensive guide and setup documentation, allowing developers to explore and implement this new approach to AI instruction following.