Black Hat Asia keynote assesses practical limits of agentic offensive security automation

At Black Hat Asia in Singapore on 27 April 2026, RunSybil CEO Ari Herbert-Voss presented a keynote examining the real-world impact of frontier LLMs like Anthropic's Mythos and OpenAI's GPT-5.5 on offensive security. While acknowledging that the capability ceiling for autonomous exploitation is rising rapidly, Herbert-Voss argued that the capability floor—the ability to validate and operationalize findings—is not keeping pace. Evaluations of Mythos by the UK AI Security Institute show models can complete substantial portions of attack chains in controlled environments, but consistency on real-world targets remains limited. Herbert-Voss drew parallels to fuzzing in the 2000s: automated tools generate massive datasets and possible bugs, but human expertise is still required to filter, validate exploitability, and understand root causes.

LLMs demonstrate 'massive gains' in discovering and exploiting low-severity 'shallow bugs,' modest gains for mid-tier vulnerabilities, and sparse gains for the most severe flaws. The attack vector is not a specific exploit but rather the systemic use of AI to accelerate bug discovery and multi-step attack execution. Organizations should prepare for accelerated time-to-exploitation: between 2023 and 2026, the average time from bug discovery to exploitation dropped from five months to 10 hours in professional CTF environments. However, real-world deployment still requires human orchestration, particularly for complex, high-impact vulnerabilities.

All software systems are theoretically exposed to accelerated bug discovery via LLM-assisted fuzzing and exploit generation. Organizations shipping software are most at risk, as the window for patching vulnerabilities before exploitation shrinks rapidly. AI infrastructure itself (model inference servers, agent orchestration platforms, MCP servers) is also a target class.

Herbert-Voss emphasized that 'shifting left' is more important than ever—organizations must integrate security testing earlier in the development lifecycle, as the window between bug introduction and exploitation is collapsing. Invest in automated validation pipelines that can triage AI-generated bug reports and prioritize exploitable findings. Security teams should adopt AI-assisted tooling themselves to maintain parity with attackers. For high-risk systems, consider deploying AI red teams internally to stress-test defenses before external adversaries do. Finally, recognize that agentic offensive security is a teachable moment: the gap between capability and operationalization is where human expertise remains indispensable—hire and retain skilled security engineers who can validate and contextualize automated findings.