Rust Firewall Scans LLM Prompts in 12 Microseconds

Local LLM Security: The Need for Speed

Large language model applications face two critical security and privacy hurdles. First, users often inadvertently or maliciously expose sensitive data like Social Security numbers, credit card details, or API keys within their prompts, which then risk being sent to third-party LLM providers. Second, attackers can employ prompt injection techniques, instructing the LLM to disregard its original directives and execute unintended actions. Standard solutions struggle to keep pace. Microsoft Presidio, for example, requires approximately 200 milliseconds per scan, while LLM Guard demands over 300 milliseconds and a heavy dependency on 47+ Python libraries, including PyTorch. Paid cloud APIs like Lakera, while effective, necessitate sending user data outside of an organization's infrastructure.

This performance gap and data privacy concern drove the development of promptfirewall, an open-source solution built in Rust. Its core objective is to provide both Personally Identifiable Information (PII) detection and prompt injection detection in under 1 millisecond, operating entirely locally with no external network calls. This approach ensures data stays within the user's control and significantly reduces latency.

The performance metrics for promptfirewall are compelling. In a scenario involving the detection of SSNs and credit card numbers, the latency recorded was a mere 12 microseconds. When testing for prompt injection, specifically the 'ignore previous instructions' type, the latency was similarly low, at 15 microseconds. Even a combined scan, testing for both PII and injection vulnerabilities, clocked in at just 27 microseconds. These figures represent a substantial improvement over existing Python-based solutions, which often measure their performance in hundreds of milliseconds.

The architectural choice of Rust is central to achieving these speeds. Rust offers memory safety without a garbage collector, enabling predictable performance and low-level control crucial for high-speed operations. This contrasts with Python, whose Global Interpreter Lock (GIL) and dynamic typing can introduce overhead and performance bottlenecks in computationally intensive tasks. By leveraging Rust's performance characteristics, promptfirewall bypasses the typical latency issues associated with LLM security checks, making it a viable option for real-time, high-throughput applications.

Under the Hood: How Promptfirewall Works

Promptfirewall's design prioritizes efficiency and local execution. For PII detection, it employs a combination of regular expressions and a custom-built, highly optimized Finite State Machine (FSM). This FSM is trained to recognize common patterns associated with sensitive data, such as credit card number formats and SSN structures. The speed of this approach is further enhanced by Rust's efficient string processing capabilities and the ability to compile down to highly optimized machine code.

The prompt injection detection mechanism is equally focused on speed and accuracy. Instead of relying on complex natural language understanding models that would introduce significant latency, promptfirewall uses a curated set of keywords and pattern matching rules. These rules are designed to identify common phrases and structures used in prompt injection attacks, like "Ignore previous instructions," "You are now a...", or variations thereof. The system maintains a dynamic list of known injection patterns, allowing for updates without requiring a full recompile of the core engine. This approach is akin to a highly tuned spell-checker, but for malicious instructions.

Crucially, promptfirewall operates as a standalone library. Developers can integrate it directly into their LLM application backend. This means that before any prompt is sent to an external LLM API, it is first processed by promptfirewall. If the scan detects PII or a potential injection attempt, the application can then take predefined actions, such as blocking the prompt, redacting sensitive information, or alerting the user, all without the data ever leaving the user's environment or incurring external API costs.

The choice to build this tool in Rust was deliberate. Unlike Python, which often requires extensive libraries and can be slower for CPU-bound tasks, Rust provides low-level control and performance comparable to C/C++. This allows for extremely fast parsing and pattern matching. The absence of a garbage collector in Rust also means more predictable performance, without the pauses that can occur with garbage collection cycles in other languages. This is critical for applications demanding sub-millisecond response times.

The project is currently available as a Rust library, with plans for potential bindings to other languages in the future. The developer, Tim, has open-sourced the project on GitHub, encouraging community contributions and further development. The goal is to provide a robust, fast, and accessible security layer for the rapidly growing LLM application ecosystem.

The Broader Implications for LLM Development

The availability of a local, high-speed firewall like promptfirewall shifts the landscape for LLM application development. Developers no longer need to choose between data privacy and security, or between performance and protection. This tool enables them to implement essential safeguards directly within their application's architecture, maintaining full control over user data and ensuring rapid response times.

Consider an application handling customer support queries via an LLM. Without a fast, local scanner, sensitive customer information shared in a chat might be sent to a third-party LLM provider, creating a privacy risk. Furthermore, a malicious user could attempt to inject commands to extract other customer data or manipulate the chatbot's behavior. Promptfirewall addresses both issues by scanning the prompt locally in microseconds. If PII is detected, it can be masked or removed before reaching the LLM. If an injection attempt is recognized, the prompt can be rejected outright, preventing potential data breaches or system abuse.

The open-source nature of promptfirewall also democratizes access to advanced security features. Previously, robust PII detection and prompt injection defenses often involved costly third-party services or complex, resource-intensive custom implementations. Now, developers of all sizes can integrate this high-performance security layer into their applications at no direct cost, fostering a more secure and trustworthy ecosystem for LLM-powered products. This is particularly impactful for startups and smaller teams who may not have the resources for expensive enterprise security solutions.

What remains to be seen is how quickly the community will adopt and extend promptfirewall. While the current implementation covers common PII types and injection vectors, the evolving nature of LLM attacks and data privacy regulations may necessitate continuous updates and enhancements. The true long-term value will depend on community contributions and the tool's adaptability to new threats.