The Problem: Unconstrained AI Agents and System Access

We are rapidly moving towards AI agents that not only understand and generate text but also execute commands on our systems. While prompt injection attacks often dominate the security conversation, the more immediate and potentially devastating threat lies in granting these agents unfettered access to system resources like the shell. A few weeks ago, an LLM-driven coding agent with shell access on a disposable server attempted to curl a metadata endpoint, write files outside its designated directory, and read shell history within ten minutes of unconstrained experimentation. These actions weren't malicious in intent; the model was simply exploring its capabilities in a broad, unguided manner. This highlights a critical gap: we focus on the text-based prompt injection problem while overlooking the fundamental systems-level risks of granting AI agents capabilities like filesystem and network access.

This article details a practical, reproducible harness designed to observe and assert precisely what a tool-calling AI agent can do at the syscall level. This is essential for understanding an agent's true reach before it interacts with any sensitive environment.

Diagram illustrating the flow of commands from an AI agent to a sandboxed shell environment.

Building a Secure Testing Harness

The core of any secure AI agent deployment, especially one with system access, must be rigorous testing in an isolated environment. This harness is built on a modest Linux box, ideally a free, disposable cloud server, making it perfect for adversarial experimentation. The goal is to create a minimal, reproducible setup that allows us to monitor and control the agent's interactions with the underlying operating system.

The setup involves several key components:

  • Isolated Environment: A dedicated virtual machine or container that serves as the agent's playground. This environment must be ephemeral, meaning it can be easily created, tested, and destroyed without leaving any persistent changes.
  • Command Interception: A mechanism to capture all shell commands executed by the agent. This could involve wrapping the shell itself or using system tracing tools.
  • System Call Monitoring: Beyond just commands, it's crucial to monitor the actual system calls (syscalls) the agent's processes make. Tools like strace on Linux are invaluable here.
  • Assertions and Policies: Defining clear rules about what the agent *should* and *should not* be able to do. These policies are then translated into checks that are run against the captured syscalls and commands.

Observing Agent Behavior with `strace`

strace is a powerful Linux utility that intercepts and records system calls made by a process and signals received by a process. By attaching strace to the shell process that the AI agent is interacting with, we can gain granular insight into its actions.

Consider an agent tasked with writing a file. Instead of just checking if the file was created, strace can show us the sequence of syscalls: openat to request file access, potentially mmap for memory mapping, write to put data into the file, and finally close. If the agent tries to write to a restricted directory, the openat or subsequent write syscall will likely fail with a permission denied error.

To implement this, you would typically launch the agent's shell within a strace command. For example:

strace -f -o agent_trace.log /bin/bash

The -f flag ensures that strace follows child processes, which is crucial for agents that might spawn sub-processes. The output agent_trace.log will contain a detailed, line-by-line record of every syscall.

Defining and Enforcing Policies

Once we have the syscall data, the next step is to define and enforce policies. These policies act as guardrails, specifying the boundaries of the agent's permitted actions.

A policy might include rules such as:

  • Directory Restrictions: The agent should only be allowed to write files within a specific, designated working directory (e.g., /agent_work). Any attempt to openat or creat outside this path should trigger an alert or failure.
  • Network Access Control: The agent should not be able to initiate outbound network connections to arbitrary hosts, especially to metadata endpoints common in cloud environments (e.g., 169.254.169.254). This involves monitoring syscalls like socket, connect, and sendto.
  • Command Whitelisting/Blacklisting: While syscall monitoring is more robust, a simpler layer can involve blacklisting known dangerous commands (e.g., rm -rf /, dd on sensitive devices) or whitelisting only approved commands.
  • Read Access Limitations: The agent should not be able to read sensitive system files (e.g., /etc/passwd, ~/.bash_history). Monitoring read and openat syscalls for these paths is key.

These policies can be implemented as scripts that parse the strace log. For instance, a Python script could iterate through the log file, looking for specific syscall patterns and arguments that violate the defined policy. Upon detection of a violation, the script can immediately terminate the agent's process or flag the incident for review.

The Surprise: Helpful Intent, Dangerous Execution

The surprising detail here is not that an AI agent would attempt to probe its environment, but how quickly and how broadly it would do so, driven by what it perceives as