A Go botnet called NadMesh turned up in early July hunting exposed AI services, and the operator's own dashboard claims 3,811 unique AWS keys. A Shodan harvester keeps the scan queue stocked with ComfyUI, Ollama, n8n, Open WebUI, Langflow, and Gradio: the image generators, local model runners, and workflow builders that teams stand up fast and firewall late. The intel feed behind that counter
Capital One on Thursday released VulnHunter , an open-source, agentic AI security tool that scans source code for exploitable vulnerabilities, maps out how an attacker would reach them, and proposes targeted fixes — all before a single line ships to production. The tool, built internally and now available on GitHub under an Apache 2.0 license, is one of the most ambitious attempts by a major financial institution to turn offensive AI capabilities into a public defensive resource. At a time when security teams are facing a rising tide of new AI threats, Capital One's decision to open-source the tool reflects an effort, according to CISO Chris Nims, to address "an increasingly brief window before sophisticated, next-generation AI attack capabilities become affordable and accessible to virtually every adversary." Capital One is not simply releasing another vulnerability scanner. VulnHunter introduces what the company calls an " attacker-first forward analysis " — a workflow in which the tool begins at the points where a real adversary would enter a system, such as APIs, network messages, or file uploads, and reasons forward through the application's logic to determine whether an exploit path actually survives the code's existing defenses. Conventional scanners typically work in reverse, flagging a dangerous-looking code pattern and then searching backward for a hypothetical attacker. That approach, security practitioners widely acknowledge, buries engineering teams under avalanches of false positives. VulnHunter attacks that problem head-on with a second innovation: a built-in "falsification engine" that tries to disprove its own findings before a developer ever sees them. After the tool surfaces a potential vulnerability, a structured reasoning workflow hunts for logical gaps, unsupported assumptions, and conditions that would prevent the attack from succeeding. Only findings the engine fails to rule out reach a human reviewer — and when they do, VulnHunter delivers not just an alert but a full explanation of the exploit path and a proposed code fix ready for engineering review. The tool currently runs on Anthropic's Claude Opus 4.8 model inside a Claude Code environment, though Capital One says the framework has the potential to work across other foundation models and coding harnesses. Why Capital One is giving the tool away Asked why Capital One decided to open-source a tool this consequential, Nims pointed to the communal nature of the problem. "We felt an imperative to open-source VulnHunter because modern software supply chains are very connected, and the scale of the AI threat is larger than any single organization," Nims told VentureBeat. "Securing software and our digital environments is a shared foundation that benefits developers, enterprises, and the people who depend on the systems we all build. The defensive tools to address this reality need to be just as widely distributed, tested, and improved as the codebases they protect." "Rather than wait," he added, "we decided that the right response was to build a product that is purpose-fit for today's complex security landscape, and put it into the hands of defenders everywhere." Why Capital One believes open-sourcing VulnHunter strengthens everyone's defenses The release nonetheless arrives against a backdrop the company knows well. On July 19, 2019, Capital One disclosed that an outside individual — later identified as a former Amazon Web Services employee named Paige Thompson — had gained unauthorized access to names, addresses, self-reported income, Social Security numbers, and linked bank account numbers belonging to credit card customers and applicants. The breach, which Capital One says occurred on March 22 and 23, 2019, was discovered only after an external security researcher flagged a configuration vulnerability through the company's Responsible Disclosure Program on July 17 of that year. The damage was sweeping. Approximately 100 million people in the United States and 6 million in Canada were affected. Roughly 140,000 Social Security numbers, about 80,000 linked bank account numbers, and approximately 1 million Canadian Social Insurance Numbers were compromised. The FBI arrested Thompson, and the government stated it believed the data had been recovered with no evidence of fraud. But the reputational and regulatory toll was enormous. In August 2020, the Office of the Comptroller of the Currency fined Capital One $80 million , finding that the bank had failed to adequately identify and manage risks as it migrated significant technology operations to the cloud. As Reuters reported at the time, the OCC's consent order cited insufficient network security controls, inadequate data loss prevention measures, and a board that failed to hold management accountable when internal auditing surfaced problems. The OCC also ordered Capital One to overhaul its operations and submit new cybersecurity plans for regulatory review. CyberScoop at the time called the incident " a cautionary tale for companies rushing to embrace new tech ." Capital One's own CEO, Richard D. Fairbank, acknowledged the gravity of the moment. "While I am grateful that the perpetrator has been caught, I am deeply sorry for what has happened," Fairbank said at the time. "I sincerely apologize for the understandable worry this incident must be causing those affected and I am committed to making it right." Inside Capital One's decade-long open-source strategy — and what VulnHunter adds to it. What followed was not a retreat from technology but a doubling down — with security explicitly at the center. Capital One began releasing open-source projects in 2014 and declared itself an " open-source first " company in 2015 as part of a broader technology transformation that began over a decade ago. The company has continued to invest in software supply chain security, open-source governance, and AI-driven defense. In August 2022, Capital One joined the Open Source Security Foundation as a premier member, earning a seat on the organization's Governing Board. Chris Nims, then EVP of Cloud & Productivity Engineering, framed the move as a natural extension of the company's operating philosophy. "As a highly-regulated company, we are seasoned in managing compliance and governance and advocate for standardization, automation and collaboration," Nims said in the OpenSSF announcement . Behind that public commitment lay a substantial operational apparatus. Capital One's Open Source Program Office , now in its third iteration, manages open-source usage, contributions, and community building across the enterprise. The company has released more than 40 open-source projects and has made thousands of contributions to external open-source projects it depends on, according to the company. Those efforts address not just code dependencies but the entire software development lifecycle — DevSecOps tools, infrastructure, and the collaborative environments, both internal and external, that shape how software gets built and shipped. VulnHunter is the most consequential product of that multi-year effort — and the release reflects a calculation increasingly common among large enterprises: that in security, giving away the tool is cheaper than fighting alone. "At the end of the day our goal is to make this product as broadly accessible as possible," Nims said. The company argues that modern software supply chains are so deeply interconnected that a single vulnerability in a widely used open-source component can cascade across thousands of enterprises simultaneously. Proprietary defenses, no matter how sophisticated, cannot address a problem that is fundamentally communal. By releasing VulnHunter under a permissive license, Capital One invites the global security research community to stress-test, extend, and improve the tool — effectively crowdsourcing its own defense infrastructure while strengthening the broader ecosystem. Inside VulnHunter's three-stage AI engine for finding exploitable code For engineering leaders evaluating VulnHunter , the technical architecture is where the tool's ambitions become concrete. The workflow unfolds in three distinct stages. In the first stage — attacker-first forward analysis — VulnHunter begins at the points where an external adversary would interact with a system: API endpoints, network message handlers, file upload interfaces. From each entry point, the tool reasons forward through application logic, tracing data flows, transformations, and internal security checkpoints to determine whether an attacker can actually reach a dangerous code path. This approach mirrors how a skilled penetration tester would probe a system, but automates the process at a scale no human team could match. The second stage is where VulnHunter departs most sharply from conventional scanners. After identifying a potential vulnerability, the falsification engine runs a structured reasoning workflow designed to disprove its own conclusion. It searches for assumptions that do not hold, logical gaps in the exploit path, and environmental conditions that would prevent an attack from succeeding. Findings that fail this internal challenge are discarded before any developer sees them. Capital One's explicit goal is to shift the developer's burden away from triaging false alarms — a perennial pain point that erodes trust in security tooling and slows development velocity. In the third stage, vulnerabilities that survive the falsification engine trigger an evidence-backed remediation workflow. VulnHunter gathers supporting evidence across the codebase, maps the complete surviving exploit path, explains the defect and the specific capabilities an attacker would gain, and generates targeted code changes for engineering review. The output is not a generic advisory but a concrete, context-aware patch proposal. Capital One says it validated VulnHunter internally before release, running it across thousands of repositories spanning tens of business areas. The company reports that the tool identified and remediated vulnerabilities with speed and efficiency that far exceeded what its teams previously achieved through manual triage. Why AI-powered attacks are forcing banks to rethink traditional cyber defenses VulnHunter arrives at a moment when the cybersecurity landscape is shifting beneath the feet of every enterprise. Capital One's announcement frames the urgency in stark terms: advanced AI models have "dramatically lowered the barrier for bad actors to discover and exploit vulnerabilities in software," and the window before sophisticated AI attack capabilities become affordable and accessible to virtually every adversary is shrinking rapidly. "Safeguarding information is essential to our mission and our role as a financial institution," Nims told VentureBeat. "We have invested heavily in cybersecurity and will continue to do so to stay ahead of today's evolving threat landscape." The company's own AI security researchers have been tracking these trends closely. At NeurIPS 2024 in Vancouver, Capital One's team presented research and curated a list of nearly 100 papers spanning LLM safety, adversarial resilience, jailbreak attacks, and synthetic data generation. The papers they highlighted — including work on multi-agent defense frameworks, automated red-teaming, and guardrail classifiers — paint a picture of an arms race in which offensive and defensive AI capabilities are co-evolving at breakneck speed. Several of those research themes map directly onto VulnHunter's architecture. The falsification engine echoes the adversarial defense strategies explored in papers like " BackdoorAlign ," which demonstrated that embedding a structured safety mechanism into a small number of training examples could recover a model's safety alignment without degrading performance. The attacker-first forward analysis reflects the philosophy of " WildTeaming ," a framework that collects and analyzes real-world jailbreak attempts to build more resilient models. And VulnHunter's emphasis on minimizing false positives parallels the goals of "GuardFormer," a guardrail classifier that outperformed GPT-4 on safety benchmarks while running 14 times faster. The thread connecting all of this work is a conviction that traditional, reactive security — monitoring networks, patching known vulnerabilities, responding to incidents after they occur — is no longer sufficient when adversaries can use AI to discover and exploit zero-day vulnerabilities at machine speed. The only durable defense, Capital One argues, is to find and fix the vulnerabilities in your own code before attackers find them first. What Capital One's cloud security journey reveals about the entire banking industry Capital One's cloud journey also illuminates a broader reckoning across financial services. When Capital One moved aggressively to Amazon Web Services in the mid-2010s, it was a rarity among major banks. Most financial institutions simply did not trust third parties to store their most sensitive data. Capital One's CIO at the time, Rob Alexander, publicly championed the cloud as more secure than the bank's own data centers — a claim that the 2019 breach complicated considerably. The CyberScoop report from that period captured the tension within the industry. W. Patrick Opet, managing director of cybersecurity at JP Morgan Chase, described a cultural shift in banking from prioritizing traders to prioritizing developers: "Now, it's 'Focus on the developer, turn everything into code, and automate everything.'" Mark Nicholson, Deloitte's cyber leader for the financial industry, noted that the pressure to move quickly was exposing "weaknesses in the development methodology." And the breach itself was a reminder that even as Chase spent $600 million annually on cybersecurity, relatively simple vulnerabilities — like the Apache Struts bug that enabled the Equifax breach — could undercut massive investments in data protection. Seven years later, the industry has largely followed Capital One into the cloud, and the security challenges have only intensified. The question is no longer whether to use cloud infrastructure but how to secure the software that runs on it. VulnHunter represents part of Capital One's answer to that question: rather than relying solely on network-level controls and perimeter defenses, push security directly into the code itself, at the moment it is written. The open-source release also carries implicit competitive pressure. If VulnHunter gains traction among developers and security teams, it could set a new baseline for what enterprise security tooling is expected to do — and force rival banks, fintechs, and cloud providers to match or exceed its capabilities. Whether VulnHunter lives up to that ambition will depend on adoption, community engagement, and the tool's real-world performance against the increasingly sophisticated AI-powered attacks it was designed to counter. But the release itself tells a story that extends well beyond any single tool or any single company. In 2019, a misconfigured web application firewall — exploited alongside an overly permissive cloud access role — exposed the records of roughly 100 million Americans , drew an $80 million regulatory fine , and became one of the most widely cited cloud security incidents in the industry . In 2026, the same institution is open-sourcing an AI-driven defense built for a new generation of threats — and betting that the best way to protect its own code is to help the entire industry protect theirs.
Intuit was an early pioneer in the usage of agentic AI, but its path to success has hardly been a straight line. At VB Transform 2026 , Intuit VP of AI Nhung Ho described how the company rebuilt its agent architecture twice in the span of about four months, first moving from a fleet of specialist agents to a central orchestration layer, then abandoning that layer for a skills and tools based system once the orchestrator itself started failing under its own complexity. The full second rebuild took 60 days, with a first working version in under 20. The failure mode that forced the second rewrite was specific. Agents in the orchestrated system passed results to each other in natural language, and each handoff lost context the next agent needed to act correctly. "If you have 10 agents and they all are passing to each other, every time that pass happens, error compounds," Ho said. Why the orchestration layer broke down Ho said the original push toward specialist agents came from a straightforward customer complaint. A fleet of capable agents is still something a customer has to manage, deciding which agent to use for which task. Intuit's answer was a system that could take a task and route it internally, without asking the customer to pick an agent themselves. That orchestration layer held up for about three months, which Ho described only half joking as roughly a year in the compressed timeline of agent development in 2026. It broke for a structural reason rather than a capacity one. Passing outcomes between agents in natural language meant each downstream agent had to infer how the upstream agent reached its conclusion, and that inference degraded with each additional hop. A ten agent chain did not fail occasionally, it compounded errors by design. That diagnosis is what sent Intuit back to a skills and tools architecture. The 60-day rebuild, and what it took to get engineering buy-in Rebuilding a production agent system in 60 days required more than an architectural decision. Ho said the harder problem was internal, convincing both leadership and the engineers who had built the original agents that scrapping recent work was the right call. The pitch to leadership relied on evidence rather than argument. Ho's team built a demo of the new architecture using real customer queries pulled from production, then showed it performing better than the existing system on the same tasks. "The best proof, at least my belief, is what are customers trying to do? And whatever system you build needs to address those problems," Ho said. Winning over engineering required a different case. Hundreds of engineers outside Ho's core team had built the specialist agents being retired, and the ask was to take their agents apart into individual skills and tools instead. Ho said the motivating argument was scale. A standalone agent solved one narrow problem, while a shared skill or tool built into the new architecture could serve every customer who touched that part of the product. That shift also changed what partner teams were responsible for day to day, moving their focus from building agents to running evals, since evals became the only way to measure whether the new architecture was actually working. Bringing a human into the loop, and feedback at a different scale The clearest customer facing result of the rebuild is a feature that lets a live agent conversation pull in a human — though it's currently in early testing, live to about 1% of Intuit's customer base. "We're going to be scaling it up in the next few weeks," she said. Ho said a customer can bring in an Intuit product support person mid conversation, or their own accountant, or one of Intuit's own bookkeepers, and that person joins with the full context of what the agent has already done. Ho drew a direct contrast with how most AI chat products handle the same situation. A general purpose assistant answering a tax question typically ends with a disclaimer to consult a professional. Intuit's system is built to connect the customer to that professional directly, inside the same conversation. That human handoff sits alongside a permissions model built for financial data specifically. Every action an agent takes on a customer's financial data requires explicit permission first, though Ho said that requirement can ease over time as customers build trust in the system. Intuit keeps an audit log of everything an agent does that can be reversed if needed. Feedback in the agentic AI era The rebuild also changed how Intuit gathers and uses feedback, a shift Ho said is qualitatively different from what came before. "Feedback in the past used to be very, very sparse, and it was also very bimodal," Ho said. "Either they loved it or they hated it, and usually it tends towards the negative." In a chat based system, every conversation functions as feedback, which Ho said moved the company from roughly 0.3% of customers ever giving explicit feedback to something close to 100%. Ho said she has returned to writing code herself specifically to build models that analyze that feedback volume systematically, looking for where the system is falling short at a scale no manual review process could keep up with. That volume comes with a tone most product teams aren't used to hearing directly. Customers tell the agent exactly where it failed, in plain terms. "They straight up tell you, 'You suck. I hate this. This is not right,'" Ho said. "But they're also willing to give the systems grace and correct it as well, and so the onus is on all of us to harvest this new piece of feedback and type of feedback, and actually improve the system."
Legacy infrastructure, not the models themselves, is what's actually slowing AI agents down. That was the shared conclusion of three infrastructure leaders — from LinkedIn, Walmart, and Zendesk — at VB Transform 2026 . The panel brought together Animesh Singh, senior director of AI platform and infrastructure at LinkedIn, Desiree Gosby, SVP of corporate technology services and technology strategy at Walmart, and Sami Ghoche, VP of applied AI at Zendesk, each describing what actually broke when they moved agents from pilot to production. Each arrived at the same conclusion from a different starting point: None of the bottlenecks they hit were model problems. What tied their answers together was a shared premise: most enterprise infrastructure was built for how humans work, not for how agents work. The gap between those two speeds is where the real engineering happened. Gosby put it plainly when asked what she'd learned scaling agents inside Walmart's own workforce. The goal, she said, is to make sure "engineering doesn't once again become the bottleneck for what it is we're trying to do." Where the bottleneck actually was Each company hit a different version of the same wall: infrastructure designed for how people work doesn't hold up once agents are doing the work instead. At LinkedIn, the first bottleneck wasn't a model, it was Kubernetes, which assumes containers spin up on demand, a process that takes seconds. Singh said that's too slow for agents. The fix was moving from on-demand provisioning to pre-provisioned pools of containers that swap agentic workloads in and out in real time. A second, harder problem surfaced once LinkedIn let agents control their own orchestration. A five-point evaluation system looked clean, but hallucination kept showing up anyway. Singh said the issue was structural, an LLM evaluating another LLM's output shares the same failure mode as the thing it's evaluating. "We built our own harness, our own control flow, and pushed the LLMs to the leaf instead of them orchestrating the loop," Singh said. Roughly 80% of the workflow is now scripted, deterministic code, with LLMs used only where reasoning is required, and each step's evidence is committed to disk before the system moves on. Walmart's bottleneck came from success. An agent harness put directly into employees' hands went viral internally, and what Gosby called "citizen developers" began building their own agents to solve problems that once required a formal engineering roadmap. The upside was real innovation. The downside was duplication, dozens of overlapping agents with no coordination. The fix wasn't reining in the harness, it was building governance to spot duplication, promote the best version of an agent, and get it into production without engineering becoming a chokepoint. Zendesk hit its bottleneck from the data side. Ghoche, who joined through Zendesk's acquisition of Forethought , which closed in March 2026, described sitting on what he called a public figure of 20 billion customer conversations in Zendesk's repository. The instinct is to hand that history to a large language model with a big context window and let it generate the agents a business needs. Ghoche said that doesn't work. "You can't really do that, so instead you have to really invest in the underlying data pipelines and all the data infrastructure that comes with that," he said. The role of open source On open source, all three leaders landed on a similar instinct: own what you can, and lean on frontier labs only where they still have a clear edge. Ghoche said his own view is that most enterprises would prefer to own their models and infrastructure wherever that's possible, and that reasoning is what drives Zendesk's own approach. The exception is frontier reasoning work, where the labs still lead, though he said that slice of use cases is shrinking relative to everything else enterprises now do with AI. LinkedIn's answer was to build two subsystems specifically for independence. The first is what the company calls an AI gateway, a single interface that every outbound call to a model runs through regardless of provider. The second component is a memory subsystem built to hold context independent of any model provider. "Every single outbound call going to an LLM, whether it's on a public cloud or on-prem in our own data centers, follows the same semantics, the same API calls. We can quickly switch between different providers," Singh said. Walmart built its own internal gateway to stay vendor agnostic across three workload types: fully deterministic workflows, planner-and-reasoner workflows for open-ended tasks, and a hybrid of the two. Compliance-heavy work stays deterministic by design; governance, security and evaluation run through the gateway regardless of which model is on the other end. Gosby said the choice between a frontier model and an open-weight model comes down to whichever is most effective for the specific workload, not a fixed policy. Advice for the modernization journey Three pieces of advice came up directly, each tied to the wall a leader had already hit. Invest in evals before anything else. Ghoche called it the thing common to every use case, internal or customer facing. "The thing that's common to all of these is evals. It'll force you to break the problem down, and once you have a robust set of evals, you can move a lot faster," he said, Own your agent harness from day one. Gosby's advice was to put the AI harness directly in employees' hands early, paired with the infrastructure to monitor what it produces. "It will unlock a huge amount of innovation," she said. Build for model and context independence. Ensuring flexibility is critical for success. "Build for independence, whether it's a frontier model of today versus an open source model of tomorrow," Singh said. "Keep that context within your enterprise so that you can reuse it when you ship the model or the harness tomorrow," Singh said.
OpenClaw has become one of the most widely adopted agentic frameworks, but it has yet to prove itself at enterprise scale. Agents need real credentials — API keys, OAuth tokens, service accounts — to work effectively, and Brex found that traditional guardrails couldn't contain what those agents were doing with them. Brex set out to overcome these limitations by building an internal platform it calls CrabTrap. The open-source HTTP/HTTPS proxy intercepts all network traffic, examines policy rules, and uses a LLM-as-a-judge to decide whether agent requests should be approved or denied. “What we noticed was that the network layer was an untapped enforcement point,” Brex co-founder and CEO Pedro Franceschi told VentureBeat. “Every request an agent makes is an opportunity to intercept, reason about, and make a policy decision.” The takeaway Franceschi wants IT leaders to draw: agent governance should shift from SDK-level permissions and model guardrails toward a centralized network control plane that enforces and learns from real in-the-wild agent behavior. How Brex targeted the transport layer The “obvious fix” (at least initially) to the agent security gap was guardrails, and much of the early work has centered on scoped tools, per-action permissions, and human-in-the-loop approvals. But as agents evolve, each new capability means there’s another API to tune or surface to audit, Franceschi noted. “Any agentic system with multiple tools and access to the open internet creates an immediate tension for builders: The more capable you make an agent, the more dangerous it becomes, and the safer you make it, the less useful it is,” he said. Existing solutions to this tradeoff were “weak”: Fine-grained API tokens help at the margins but can still be misused and constrain functionality. Semantic guardrails (such as context, skills, or prompt steering) are easily bypassed by prompt injection, especially for agents connected to the internet. Agents can be “defanged” when given read-only access or limited toolsets, but then they can't do meaningful work, Franceschi said. On the other hand, granting broad write access and a large tool surface can result in hallucinations and real production consequences. Model context protocol (MCP) gateways enforce policy at the protocol layer — but only for traffic using MCP. Meanwhile, guardrails from LLM providers are tied to a single model and can be “opaque” to customize with enterprise-specific policies. And powerful tools like Nvidia OpenShell offer more of a “per-sandbox egress control.” “When we started, we hadn’t found a solution to deploying harnesses like OpenClaw safely,” Franceschi said. “Instead of waiting for the industry to catch up, we decided to own the problem and invent the necessary tools.” Notably, they needed a platform that sat between every agent and every network request, and could make “nuanced decisions about what to allow,” he said. This made the transport layer a core architectural component and natural starting point, he said. By operating at this layer, CrabTrap is framework-agnostic, language-agnostic, and API-agnostic. It doesn't require SDK wrappers or per-tool integration. Users set HTTP_PROXY and HTTPS_PROXY in the agent's environment, and every outbound request routes through the proxy before it reaches a destination. However, Franceschi emphasized, Brex didn't start at the transport layer because it thought it was the only answer; rather, they believe in “security by layers.” “The transport layer was simply an underinvested one, and we saw an opportunity to add meaningful enforcement there alongside everything else,” he said. The LLM-as-a-judge training loop CrabTrap combines deterministic static rules with an LLM-as-a-judge for requests that fall outside known patterns, Franceschi explained. The judge only “fires on the long tail of unfamiliar endpoints or unusual request shapes,” which for a mature agent is typically fewer than 3% of requests. The more pressing problem was how to know that a policy is the right one? With static rules, it's “relatively straightforward” to reason about accuracy. But with an LLM judge, the system is nondeterministic, and users need confidence that the policy approves the right requests and blocks the rest. “Our key insight was to bootstrap policy from observed behavior rather than write it from scratch,” Franceschi said. Beginning with real behavior and editing down based on real-world learnings turned out to be “dramatically more effective than starting from a blank page.” Brex’s team built a policy builder (itself an agentic loop) that runs underlying agents in shadow mode, analyzes historic network traffic, samples representative calls, and drafts a natural-language policy that matches what the agent actually does. From there, they built an eval system that tests policy changes before they go live. CrabTrap compares historical audit entries against a draft policy and reports the exact changes to be made. Users can slice results by method, URL, original decision, and agreement status. All of this runs with concurrent judge calls, so replaying thousands of requests “takes minutes, not hours,” Franceschi said. Brex also developed a live feedback loop: Full audit trails are stored in PostgreSQL and queryable through the admin API and dashboard. In cases where a resource is continuously denied, the system can notify a human or an agent to propose a policy update for review. “That closes the loop between observed denials and policy refinement,” Franceschi said. Core challenges and roadblocks Of course, the build wasn’t without its challenges. A big one was latency: “Putting an LLM between an agent and every outbound API request sounds like it would grind things to a halt,” he said. However, it didn’t turn out to be as big a problem as expected. This was for two reasons: The LLM judge only activates on a small fraction of requests (the aforementioned 3%). Agents quickly settle into predictable traffic patterns; once observed, high-volume patterns become static rules. Second, by using small, fast models like Claude Haiku meant that, even when the judge did fire, added latency was “negligible.” This can be further reduced with local models and prompt caching, Franceschi said. The harder and less obvious challenge was prompt injection, he said. The judge receives the full HTTP request and all content is user-controlled, so potentially, a crafted URL, header, or request body could manipulate the judge's decision. Brex addressed this by structuring the request as a JSON object before sending it to the model, so all user-controlled content is “escaped rather than interpolated as raw text,” Franceschi said. Results, and where CrabTrap might evolve Brex tracks a few factors to measure CrabTrap’s internal impact: Engagement with agents, network traffic patterns, and net promoter scores (NPS). The most meaningful result of CrabTrap has been “organizational confidence,” Franceschi said. Previously, the team had “real hesitation” when it came to deploying autonomous agents broadly across business operations, because the existing guardrail options didn't provide enough assurance. “CrabTrap changed that calculus,” Franceschi said. They now have an enforcement layer they trust, increasing confidence around expanding agent deployment into more parts of the business and delegating more agent configuration and management to users. Franceschi described the policies derived from traffic as “surprisingly strong.” The team expected the policy builder to produce a “rough starting point” requiring heavy manual editing. In practice, though, pointing the platform at a few days of real traffic produced policies that matched human judgment on the “vast majority of held-out requests.” Additionally, CrabTrap revealed how much noise agents generate. “The audit trail made this visible for the first time,” Franceschi said. They used denial logs and traffic analysis not only to tune policies, but to tighten agents themselves, remove tools, and cut out entire categories of requests that were wasting both time and tokens. “The proxy became a discovery tool, not just an enforcement one,” he said. Areas for growth (and input from the open-source community) Brex anticipates CrabTrap to continue to evolve, particularly as they have released it as open-source. “We hope the community helps shape it,” Franceschi said. Areas of improvement include deeper authentication functionality such as single-sign on (SSO), fine-grained role-based access control (RBAC); escalation workflows that allow agents to request additional permissions; and policy recommendations based on denial patterns. Programmatic configuration, or developing API endpoints for “creating, forking, and applying” policies to agents, could allow the whole policy lifecycle to be automated rather than managed manually, Franceschi said. As for escalation, if an agent is continuously denied a given resource or endpoint, it should be able to route requests to humans or other AI agents for review and back that up with a rationale for why it needs access. “That turns CrabTrap from a hard enforcement boundary into something more like a managed permission system,” Franceschi said. Additionally, the policy was built to bootstrap from network traffic, but there is opportunity to incorporate additional signals around agent traces and resource-calling, as well as broader context on what agents are ultimately trying to accomplish. This can help produce more accurate and nuanced policies. Finally, there's an “open philosophical question” about the right posture for CrabTrap: Should it be a fully transparent layer that the agent itself is unaware of, or should it operate more like a “well-intentioned manager”? (that is, the agent knows about the layer and can interact with it). The open-source community can help shape these developments, and CrabTrap will only get better with more users, Franceschi said. Brex’s agents speak to a specific set of APIs; teams using CrabTrap with different agents, services, and policy requirements will surface “edge cases and patterns we can't hit alone.” “We have ambitious plans for where it could go, and we’d rather build in the open,” Franceschi said. What other builders can learn from CrabTrap The response has been stronger than expected. CrabTrap has more than 700 stars on GitHub . Franceschi said Brex has also heard from OpenAI, Y Combinator CEO Garry Tan, and programmer Pete Steinberger, all expressing interest in deploying similar internal infrastructure. The broader lesson: “Don't let infrastructure gaps become excuses to wait," Franceschi advised. There are “real blockers” for every enterprise looking to seriously deploy AI agents, including security concerns, lack of tooling, or unclear guardrails. “It's tempting to sit on your hands until the industry catches up,” he said. “The lesson from CrabTrap is that you can own those problems directly.”
