Source: https://blog.cloudflare.com/cyber-frontier-models/ Published: May 17, 2026 (Cloudflare engineering blog) Authors: Albert Pedersen, Craig Strubhart, Dan Jones, Irtefa Fairuz, Martin Schwarzl, Rohit Chenna Reddy
A first-hand engineering report by Cloudflare on its testing of Claude Mythos Preview against more than 50 of its own production repositories under Project Glasswing: Securing Critical Software for the AI Era. It is a public technical comparison between Mythos and previous-generation frontier models on real production codebases, and documents two operative new capabilities, the inconsistency of the model's organic refusals, the signal-to-noise problem in AI vulnerability research, and a four-lesson "harness" architecture Cloudflare built around the model. The repositories tested spanned Cloudflare's runtime, edge data path, protocol stack, control plane, and open-source dependencies. Where the Project Glasswing: Securing Critical Software for the AI Era page describes the Anthropic side of the partnership, this write-up provides the partner-side technical and operational evidence, and serves as independent corroboration of Anthropic's Claude Mythos Preview capability claims from a technical partner.
Capabilities relative to previous models
Cloudflare frames Mythos Preview as "a real step forward… not just a refinement of what came before." Two features are singled out:
- Exploit chain construction. Real attacks chain multiple small primitives. Previous frontier models could find the underlying bugs and describe why they mattered but stopped before stitching them into a chain. Mythos Preview can take low-severity bugs that would traditionally sit invisible in a backlog and chain them into "a single, more severe exploit." Cloudflare writes that the reasoning "looks like the work of a senior researcher rather than the output of an automated scanner."
- Proof generation. Mythos writes code that would trigger the suspected bug, compiles it in a scratch environment, runs it, reads the output, and adjusts its hypothesis if the result doesn't match. The model "closes the gap" between suspected flaw and working proof-of-concept (PoC) autonomously.
Organic-refusal inconsistency
The Mythos Preview version Cloudflare used "did not have the additional safeguards present in generally available models (like Opus 4.7 or GPT-5.5)." Despite this, the report states the model "organically pushes back on certain requests," exhibiting guardrails alongside its cyber capabilities, but that these organic refusals are not consistent. The examples given:
- The same task, framed differently, produced opposite outcomes.
- The same code, with only an unrelated environment change, produced a refusal in one case and acceptance in another.
- The same finding (confirmed memory bugs) produced a refusal to write a demonstration exploit, then acceptance on a re-framed request.
- Even identical requests produced different outcomes across runs due to model stochasticity.
Cloudflare's conclusion: "while the model's organic refusals/guardrails are real, they aren't consistent enough to serve as a complete safety boundary on their own. That's precisely why any capable cyber frontier model made generally available in the future must include additional safeguards on top of this baseline behavior."
The signal-to-noise problem
Cloudflare identifies two factors as dominant in the AI vulnerability-research noise rate:
- Programming language. C/C++ produce more false positives than memory-safe languages like Rust because the bug classes (buffer overflows, out-of-bounds reads/writes) don't exist in Rust at compile time.
- Model bias. "Ask a model to find bugs, and it will find them, whether the code has any or not. Findings come back hedged with 'possibly,' 'potentially,' 'could in theory,' and the hedged findings vastly outnumber the solid ones. That's a reasonable bias for an exploratory tool. It's a ruinous one for a triage queue."
Cloudflare reports that Mythos Preview's PoC generation improves signal-to-noise: "a finding that arrives with a PoC is a finding you can act on."
Limits of a generic coding agent
Cloudflare started with the intuition that one could point a generic coding agent at an arbitrary repository and ask it to find vulnerabilities. The report says this produces findings but fails at meaningful coverage, for two reasons:
- Context shape mismatch. Coding agents are tuned for one focused stream of work, building one hypothesis at a time. Vulnerability research is "narrow and parallel by nature" — pick one specific thing (a complex feature, a security boundary, a vulnerability class like command injection) and investigate thoroughly, then do it again for the next thing, thousands of times across the codebase. A single agent session against a 100K-line repo "can cover maybe a tenth of a percent of the surface in a useful way before the model's context window fills up and compaction kicks in."
- Throughput. "A single-stream agent does one thing at a time," but real codebases need many hypotheses against many components at once. Beyond a certain point, Cloudflare says, the limit is the shape of the interaction rather than the model.
The four-lesson harness
Cloudflare codifies the architecture-around-the-model lessons it derived:
- Narrow scope produces better findings. "Find vulnerabilities in this repository" leads to wandering, whereas "Look for command injection in this specific function, with this trust boundary above it, here's the architecture document and here's prior coverage" produces researcher-like behavior.
- Adversarial review reduces noise. A second agent placed between an initial finding and the queue — different prompt, different model, no ability to generate its own findings — "catches a lot of the noise that the first agent would miss." Cloudflare frames this as two agents in deliberate disagreement outperforming one careful agent.
- Splitting the chain across agents produces better reasoning. "Is this code buggy?" and "Can an attacker actually reach this bug from outside the system?" are two different questions, and the model is better at each in isolation than at the combined version.
- Parallel narrow tasks beat one exhaustive agent. Coverage improves when many agents work on tightly scoped questions and results are deduplicated, rather than asking one agent to be exhaustive.
Cloudflare presents this as a candidate architecture for vulnerability discovery, extending the patterns on Agent Architecture Patterns.
The "patch faster" critique
Cloudflare pushes back on the security-leader response of compressing the patch cycle:
"The loudest reaction to Mythos Preview from other security leaders has been about speed — scan faster, patch faster, compress the response cycle. More than one team we have spoken with is now operating under a two-hour SLA from CVE release to patch in production. The instinct is understandable: when the attacker timeline shortens, the defender timeline has to shorten with it. Faster is not going to be enough, and we think a lot of teams are about to spend a lot of time, effort, and money learning that the hard way."
The harder question, per Cloudflare, is what the architecture around the vulnerability should look like: making exploitation harder when a bug exists through defenses in front of the application, isolation between application parts, and simultaneous-rollout-everywhere capability. Cloudflare argues that patching faster without those architectural properties produces worse bugs (when regression testing is skipped) than the bugs being patched, citing its own experience letting the model write patches and watching "a few go out that fixed the original bug while quietly breaking something else."
Dual-use acknowledgment
Cloudflare closes with a dual-use frame: "The same capabilities that helped us find bugs in our own code will, in the wrong hands, accelerate the attack side against every application on the Internet." Cloudflare presents this as consistent with the Defensive AI Paradox framing.
Relationships
- supports: Project Glasswing: Securing Critical Software for the AI Era — partner-side technical write-up
- supports: Claude Mythos Preview — operational corroboration of capability claims
- supports: Autonomous cyber-agents — first detailed harness architecture
- supports: Defensive AI Paradox — closes with explicit dual-use frame
- supports: AI and Cybersecurity — comprehensive partner write-up
- contradicts: (none) — partner-side independent corroboration
- depends-on: Agent Architecture Patterns — harness lessons extend agent-architecture patterns
- related: Cloudflare (company — page may not exist; add if missing), Anthropic, AI Software Progress, Defensive AI Paradox, Autonomous cyber-agents