Topic
#audit workflow
Loot, blog posts and adjacent themes connected to this topic. Follow the tag to keep it in your orbit.
Loot
More from this topic
#AI agents#code review#security#Cloudflare#Mythos#audit workflow
A practical defensive guide for checking your own codebase with AI agents: narrow scopes, parallel hunts, adversarial validation, reachability tracing, dedupe, gapfill, and governance gates. Built from the core operational lessons in Cloudflare's Project Glasswing write-up. Codebase Audit Harness Guide from Cloudflare Mythos Use this only for repositories you own or are explicitly authorized to test. The goal is defensive codebase review: better coverage, lower false positives, and a cleaner path from suspected bug to fix decision. The core lesson Do not point one generic coding agent at a large repository and ask it to find vulnerabilities. That creates shallow coverage, context loss, noisy findings, and weak triage. Instead, build a harness: a repeatable pipeline that breaks the codebase into narrow tasks, runs many focused agents in parallel, validates findings adversarially, traces reachability, deduplicates root causes, and emits structured reports. The 8-stage audit harness Recon: map the system before hunting Goal: produce shared context for all later agents. Create an architecture note that includes: Repository purpose and key services Build and test commands Entry points: HTTP routes, RPC handlers, CLIs, workers, cron jobs, webhooks, message consumers Trust boundaries: user input, internal service input, admin-only input, third-party callbacks, file uploads, deserialization points Security-sensitive modules: auth, session handling, permissions, payments, secrets, network calls, shell/process execution, templating, SQL/ORM queries, file writes High-risk languages or layers: C/C++, unsafe Rust, native bindings, parsers, compression, protocol handling Output format: Task slicing: make every hunt narrow Bad task: > Find vulnerabilities in this repository. Good task: > Check command injection in src/jobs/export.ts:createArchive() where user-controlled project names cross into shell arguments. Use docs/architecture.md and only report if attacker-controlled input can influence the command. Each task should have exactly: One attack class One function, module, or boundary One input source One expected proof standard One explicit non-goal Recommended attack classes for web/app repos: Auth bypass Broken object-level authorization Command injection SQL/NoSQL injection Server-side request forgery Path traversal Unsafe deserialization Template injection Stored/reflected XSS Secrets exposure Insecure file upload Webhook signature bypass Race condition in billing, permissions, or quota Cross-tenant data access Hunt: run many small agents, not one big one Run focused hunter agents in parallel. For a small codebase, start with 5 to 10. For a larger codebase, scale up by subsystem. Hunter prompt template: Validate: use a second agent to disprove findings Never let the hunter be the final judge of its own work. Validation agent rules: It receives only the candidate finding and the relevant code scope. It cannot create new findings. Its only job is to falsify, downgrade, or confirm the candidate. Use a different prompt and, if possible, a different model. Validator prompt template: Split bug existence from reachability Ask two different questions separately: Is this code locally buggy? Can an attacker reach it from outside the system? Do not combine these in one prompt. Combined prompts produce mush: the model mixes code correctness, exploitability, and risk into one vague answer. Reachability tracer checklist: Public route, API method, webhook, CLI, worker, queue, or import path Required auth state Required role or tenant Input constraints Feature flags Deployment exposure Network boundary Rate limits or approval gates Whether the vulnerable function is actually used in production Reachability output: Gapfill: re-queue weak coverage Hunters should mark what they did not cover. Examples: Function was too large for one pass Only one branch was reviewed Tests/build failed Type definitions were missing The call graph crossed into another repo A sanitizer looked custom and needs separate review Gapfill task template: Dedupe: collapse variants into root causes AI agents will often report the same bug through multiple paths. Treat variant discovery as useful, but do not let it inflate the queue. Deduplicate by: Same vulnerable function Same missing control Same sanitizer bypass Same trust boundary mistake Same sink Same patch required Dedupe output: Report: structured data, not prose Every accepted finding should become a structured record. Minimum report schema: The minimum viable version If you do not have infrastructure for 50 agents, start here: Run one Recon pass. Generate 20 narrow tasks. Run 5 hunter agents in parallel. Validate every candidate with a separate validator prompt. Trace reachability only for validated findings. Dedupe manually. Put accepted results into a structured issue template. That is already better than one giant "scan this repo" prompt. Safety and governance controls Do not rely on model refusals as your safety boundary. Put controls outside the model: Only scan owned or authorized repositories Restrict tool access per agent role Use read-only repository access for recon and validation Run any build/test/repro work in isolated scratch environments Keep outbound network disabled unless explicitly required Log prompts, tool calls, files accessed, and outputs Require human approval before running destructive commands or publishing reports Store findings in a private tracker until fixed or disclosed responsibly Never ask public models to generate harmful exploit payloads against third-party systems Quick scoring rubric Prioritize findings that have: External or cross-tenant reachability Clear dataflow from source to sink Minimal preconditions Missing or bypassable security control Safe reproduction evidence Independent validator confirmation Deprioritize findings that are: Pure speculation Only reachable by trusted admins Blocked by existing validation Dead code Duplicates of the same root cause Missing attacker-controlled input Copyable workflow checklist [ ] Build architecture/recon map [ ] Identify entry points and trust boundaries [ ] Create narrow task queue: one attack class + one scope [ ] Run parallel hunter agents [ ] Require structured JSON output [ ] Validate every finding with a second adversarial agent [ ] Split local bug analysis from external reachability [ ] Gapfill weak coverage [ ] Dedupe by root cause [ ] Create structured reports [ ] Human-review before fix priority or disclosure [ ] Keep authorization, audit logs, and approval gates outside the model What good looks like A good AI codebase audit harness should make your security review more systematic, not more magical. The win is not that the model finds one impressive bug. The win is that the harness repeatedly turns a huge codebase into small reviewable questions, rejects noise, preserves evidence, and creates fix-ready tickets that humans can trust.
Blog
Related reads
No blog posts for #audit workflow yet
There is no published article with this tag right now. Browse the blog for adjacent themes or follow the tag for future updates.