Multi-Agent AI and Zero-Day Vulnerabilities: The HPTSA Research Explained

Why the follow-up study matters

The one-day study gave its agent a vulnerability description. The follow-up preprint, Teams of LLM Agents can Exploit Zero-Day Vulnerabilities ↗, removed that advantage and asked whether a more structured team of agents could explore, plan, and exploit vulnerabilities without being told which vulnerability to pursue.

The uploaded v2 paper, dated March 30, 2025, calls this the zero-day setting. In practical terms, the tested agent did not receive the vulnerability description. The researchers used vulnerabilities that were past the model's stated knowledge cutoff and ran them in isolated environments.

The terminology needs care. The benchmark instances had CVE identifiers by the time the researchers assembled and reported them. “Zero-day” here describes what the tested agent knew, not proof that the experiments attacked undisclosed live production systems.

HPTSA in plain language

The authors introduced Hierarchical Planning and Task-Specific Agents, or HPTSA. Instead of asking one long-running agent to explore every path, remember every failure, choose a vulnerability class, and execute the entire plan, the architecture separated the work:

• A hierarchical planner explored the environment and chose areas to investigate
• A manager selected task-specific agents and combined information across attempts
• Specialist agents focused on defined vulnerability classes or a generic web-security task
• Tools gave agents controlled browser, terminal, and file capabilities inside the sandbox
• Curated documents gave specialists focused technical context

This architecture addressed a weakness the authors observed in single-agent traces: once one approach failed, the agent often struggled to backtrack and try a materially different strategy.

What the v2 benchmark contained

The v2 paper reports 14 reproducible web vulnerabilities of medium severity or higher. They included XSS, CSRF, SQL injection, improper authorization, privilege escalation, arbitrary code execution, parameter manipulation, and information leakage.

The researchers chose vulnerabilities past the tested model's stated knowledge cutoff, limited the benchmark to reproducible open-source systems, and manually confirmed successful exploitation. The environment was sandboxed, and the agents were prevented from searching for vulnerability descriptions.

This is more realistic than a toy capture-the-flag exercise, but it is still a selected laboratory benchmark rather than a random sample of the software ecosystem.

The reported results

With GPT-4 as the backbone, HPTSA reached 42% pass@5 and 18% pass@1. The paper reports that this was 4.3 times the pass@1 performance and twice the pass@5 performance of a single GPT-4 agent without the description.

The one-day agent that received the vulnerability description remained stronger. HPTSA came within a factor of 1.8 of that upper-bound comparison on pass@5. The tested Llama 3.1 405B and Qwen 2.5 72B backbones did not successfully exploit a benchmark vulnerability in the reported setup.

Again, 42% pass@5 does not mean that one HPTSA run has a 42% chance of compromising an arbitrary website. It means at least one of five attempts succeeded on 42% of this benchmark under the paper's specific environment and evaluation.

The ablation results are the product insight

The most useful part of the paper for defensive architecture may be the ablation study. Performance fell when the researchers removed task-specific agents, removed the curated documents, or removed the hierarchical structure.

That shows the result belonged to a system, not only to a model. Planning, delegation, retrieval, tools, memory across attempts, and repeated execution all contributed.

For defenders, this expands the review surface. A multi-agent deployment needs controls for:

• The planner's authority to create or dispatch work
• The manager's ability to grant tools and pass context
• Each specialist's separate identity and permission set
• The trust level of retrieved documents
• The scope and retention of shared memory
• Validation before one agent acts on another agent's output
• Attribution for which agent requested, approved, and executed an action

The free AI Agent Security Checklist turns those questions into a private control review.

Success and failure traces both teach something

The successful case studies showed the manager narrowing later specialist runs after earlier agents explored useful pages or failed on unrelated approaches. Multiple attempts helped the system synthesize partial information and backtrack.

The unsuccessful cases involved endpoints that were not discoverable from the visible interface or documented routes, and injection paths without obvious input fields. The authors suggested that more aggressive endpoint exploration could improve future agents.

That is precisely where responsible publication matters. The paper did not release its code or prompts, citing ethical and security concerns, and reported responsible disclosure to OpenAI. CheckLink follows the same defensive boundary: explain capability and controls without publishing an offensive implementation.

Cost is falling, but capability is not uniform

The paper estimated $4.39 per GPT-4 run and $24.40 per successful exploit based on its measured 18% pass@1 and then-current API prices. Open-source model runs were cheaper but did not succeed in the benchmark.

The numbers illustrate that repeated, automated exploration can become economically accessible. They do not create a stable price for real-world compromise. Model versions, tool policies, target complexity, monitoring, infrastructure, and the need to avoid detection all matter.

A newer result adds an important counterweight

A 2026 preprint, ZeroDayBench ↗, evaluates a different defensive task: finding and patching 22 novel critical vulnerabilities in open-source codebases. Its authors report that frontier agents were not yet capable of autonomously solving the benchmark.

The studies are not contradictory. They use different tasks, datasets, models, tool environments, and success criteria. Together, they warn against two mistakes: assuming AI agents have no meaningful cyber capability, and assuming they can reliably discover, exploit, or repair every novel vulnerability.

What ordinary people should take away

Most people will never deploy a multi-agent security system. The useful takeaway is simpler: software automation can act repeatedly and at speed, but it still depends on information, permissions, and tools.

Keep software updated, use official download sources, enable platform protections, and treat urgent “security update” links from messages as untrusted until verified. If a product asks for broad access to email, files, browser activity, or business accounts, review what it can do after access is granted.

What developers should change

Developers building agentic systems should treat every new tool as an authority decision:

1. Give each agent a separate identity
2. Default to read-only access
3. Restrict write actions to narrow APIs
4. Require approval for irreversible or external effects
5. Treat webpages and retrieved documents as untrusted data
6. Validate generated commands and structured arguments
7. Scope shared memory by task, tenant, and sensitivity
8. Log tool calls, approvals, and external effects
9. Cap actions, retries, time, and cost
10. Maintain a pause, revoke, and recovery path outside the agent

NIST's tool-use taxonomy for agent systems ↗ similarly distinguishes read-only, constrained-write, and write-capable agents across trusted and untrusted environments.

The defensive service built from the research

The AI Agent Security Review reviews identity, authorization, tools, untrusted input, delegation, shared memory, approvals, logging, and incident controls. It is a manual architecture review, not a penetration test or an autonomous agent attack.

The service is intentionally bounded. It does not accept secrets or unrestricted production credentials, and it does not reproduce the studies' withheld prompts or build exploit payloads.

Bottom line

HPTSA demonstrated that architecture can amplify capability. Hierarchy helped the system plan, specialists narrowed the work, documents added context, and repeated attempts allowed backtracking. Those same components create additional trust and authority boundaries in a production AI system.

The right defensive response is not fear or hype. It is to make identity, permissions, untrusted content, delegation, approval, observability, and recovery explicit before an agent receives consequential tools. Explore the complete evidence and related tools in the AI Cybersecurity Research Hub.