Building Nano-Harness: A Lean Coding Agent
Troy Lorents built nano-harness, a coding agent in approximately 970 non-blank lines of Python, distributed across five files. The project leverages three core tools and two providers, released under an MIT license. Its primary objective was to explore the capabilities of coding agents and their underlying architectures, often referred to as harnesses. This effort was inspired by Andrej Karpathy’s work on harnesses and loops, observed by Lorents in early February before the concept gained widespread traction.
The agent was evaluated on the full Terminal-Bench 2.0 suite, a comprehensive benchmark for evaluating AI agents in terminal environments. Nano-harness achieved a score of 59.6%, translating to 53 out of 89 tasks successfully completed. This performance was measured using Claude Opus, indicating a solid but not top-tier capability in executing complex coding tasks within a simulated terminal.
An interesting facet of the development process involved an independent frontier model, GPT Sol 5.6, reviewing the agent’s code. This model provided a rating of 4/10, which Lorents found to be the most valuable feedback received. This external code review highlighted areas for improvement and bug identification, demonstrating the utility of AI-assisted code auditing even for projects built by AI developers.
Benchmarking and Auditing: Performance and Vulnerabilities
The performance metrics for nano-harness, while respectable, reveal the current state of AI coding agents. A 59.6% success rate on a robust benchmark like Terminal-Bench 2.0 suggests that while these agents can handle a significant portion of coding tasks, there remains a substantial gap for complex, multi-step, or error-prone operations. The interaction with Claude Opus, a powerful LLM, highlights that the harness architecture and prompt engineering play a crucial role in agent effectiveness, not just the underlying model.
The code review by GPT Sol 5.6, despite its 4/10 score, proved more insightful than the benchmark performance. It underscores a critical point: human developers, or even other advanced AI models, can identify subtle bugs, inefficiencies, and potential security flaws that raw performance benchmarks might miss. This aligns with the broader trend of using AI to improve the software development lifecycle, including testing and security analysis.
The Security Implications of Local Shell Access
The existence of any coding agent capable of executing commands on a user’s machine, even one built with good intentions like nano-harness, immediately brings serious security considerations to the forefront. As highlighted by Agentik.net’s audit of their own agent runtime, agentproto, a local daemon that allows agents to run commands, read files, and open tunnels is essentially a remote shell on localhost.
This architecture inherits the threat model of any localhost daemon that exposes command execution. Potential attack vectors include browser-driven exploits, DNS rebinding attacks, and the classic insecure pattern of piping scripts directly from the internet into a shell (curl | bash). The history of web security is replete with examples of similar vulnerabilities, often stemming from seemingly innocuous features like permissive CORS settings or endpoints that execute arbitrary shell commands, as seen in the opencode teardown by wren.wtf.
When an AI agent has the ability to interact with the host system’s shell, it creates a powerful but dangerous tool. A compromised agent, or even an agent making unintentional mistakes due to flawed reasoning or prompt injection, could lead to severe data loss, unauthorized system modifications, or the exposure of sensitive information. The threat actor doesn’t need to break into the machine directly; they only need to find a way to influence the agent’s commands. This is analogous to a trusted administrator account being used by an untrusted entity.
The security implications are profound: any website visited could potentially exploit a vulnerable agent to gain control of the user’s machine. This is not a hypothetical scenario; it's a known class of vulnerability that has affected web applications for years. The addition of AI agents to this equation amplifies the risk, as the agents themselves can be manipulated or their outputs misinterpreted, leading to unintended and potentially catastrophic command execution.
What This Means for Developers and the Future of Agents
The development of nano-harness and the subsequent security audit of similar systems reveal a dual-edged sword in the advancement of AI coding agents. On one hand, agents are becoming more capable, as evidenced by their performance on benchmarks. On the other hand, the inherent power required for them to be useful—the ability to interact with development environments and systems—introduces significant security risks that must be proactively addressed.
Developers building or using such agents must approach them with extreme caution. The default assumption should be that any agent with shell access is a potential security vulnerability. Rigorous auditing, strict permission controls, sandboxing, and careful prompt engineering are not optional extras but fundamental requirements. The experience with GPT Sol 5.6's review suggests that incorporating independent AI or human code reviews into the development and deployment pipeline for these agents is essential for catching flaws before they become exploitable.
For founders in the AI agent space, the message is clear: security cannot be an afterthought. The ease with which vulnerabilities like those found in opencode can emerge, and the inherent risks of granting shell access, mean that building trust with users requires demonstrating a robust security posture. This includes transparent disclosure of capabilities, risks, and mitigation strategies. The path forward requires a delicate balance between empowering agents with necessary system access and ensuring the absolute security of user machines.