The False Positive Epidemic
Harry Wetherald at Maze highlights a critical, yet often overlooked, problem in application security: the sheer volume of false positives generated by vulnerability scanners. Estimates range from 80% to a staggering 99%. This means that for every 100 alerts a team receives, potentially 99 are not actual security risks in the context of their specific environment. The core issue isn't the potential for vulnerabilities; it's the noise generated by the tools designed to find them.
Wetherald correctly identifies that auto-fixing these findings is not only ineffective but actively harmful. Every alteration to code or cloud infrastructure, no matter how small, introduces risk. This risk can manifest as downtime, new attack vectors, or regressions in existing functionality. When 90% of the alerts are false positives, attempting to auto-fix them is akin to performing risky surgery on 90% of your patients based on a faulty diagnostic machine. The security benefit is negligible, while the potential for causing harm is significant.
The proposed solution in some circles is to improve the accuracy of these tools. However, Wetherald argues that the problem lies deeper: the tools lack contextual understanding. They flag patterns without comprehending the specific deployment, configuration, and operational environment. This is where the real challenge lies.
Focusing on the Signal, Not the Noise
The fundamental flaw in the auto-fix approach is that it treats every signal as a genuine threat. Instead, the industry needs to shift its focus from automated remediation to automated investigation and understanding. The goal should be to improve the quality of the signals presented to developers, not to blindly act on them.
Wetherald’s proposed path forward is logical and addresses the root cause of the problem. The first step is to enhance the AI's ability to investigate findings within their specific context. This means the AI should not just identify a potential vulnerability pattern; it should analyze the surrounding code, the cloud configuration, and the application's runtime behavior to determine if the pattern constitutes a genuine risk. Think of it less like a fire alarm that blares at the slightest hint of smoke, and more like a seasoned firefighter who assesses the situation before sounding the full alarm.
Once the AI can reliably determine the context and severity of a finding, the next step is to generate intelligent, human-readable fix suggestions. These suggestions should be actionable, clearly explaining the problem and how the proposed code or configuration change resolves it. This empowers developers to make informed decisions, rather than blindly accepting an automated patch.
True automation of fixes should only come after these intermediate steps have proven reliable. If the AI can accurately identify risks and generate well-understood, validated suggestions, then automating the application of those fixes becomes a viable, and indeed desirable, next step. This phased approach ensures that automation is applied where it is most effective and least risky.
The Implications for Developers and Security Teams
For development teams, this means a need to critically evaluate the security tools they employ. Are these tools generating actionable intelligence, or just a firehose of alerts? The focus should be on tools that provide context and reduce noise. Developers will need to collaborate more closely with security teams to fine-tune these tools and provide the necessary environmental context for accurate analysis.
Security professionals, in turn, must champion this shift. Instead of chasing every alert, they should focus on improving the signal-to-noise ratio. This involves investing in tools and processes that prioritize contextual analysis and intelligent investigation. The goal is to move from a reactive, alert-driven security posture to a proactive, intelligence-driven one.
The current emphasis on auto-fixing, driven by a desire for quick wins, is a misguided strategy. It introduces unnecessary risk and diverts resources from addressing genuine threats. By refocusing on understanding the signal—the true security posture within its operational context—we can build more secure systems without the inherent dangers of blind automation.
The Unanswered Question: Who Owns Context?
What remains unaddressed is the practical challenge of providing and maintaining the necessary contextual information for AI-driven investigation. Who is responsible for ensuring the AI has access to the latest code versions, deployment configurations, network topology, and runtime data? Is it the developer, the security engineer, or a dedicated platform team? The success of any context-aware security solution hinges on establishing clear ownership and robust mechanisms for feeding this critical data into the analysis pipeline. Without a clear answer, even the most sophisticated AI will struggle to accurately assess risk.