AI Findings are Cheap to Find & Costly to Trust

TL;DR 
AI has dramatically lowered the cost of producing a plausible vulnerability finding without doing the same for verifying one. For security leaders, that creates a new operational problem: more findings do not automatically produce more confidence. The scarce resource is increasingly the expert judgment required to determine what is real, what matters, and what deserves action. AI can accelerate the work, but accountability for those decisions still belongs to people.

AI is changing the economics of security in a way that is easy to miss.

Generating something that looks like a vulnerability report used to require meaningful skill and time. Today, AI can produce one in seconds. That has made it possible to search more broadly, analyze more quickly, and surface potential issues at a scale that would have been difficult to imagine only a few years ago.

But one critical cost has not fallen with it: determining whether a finding is actually true.

That distinction matters for security leaders because our problem has never simply been a shortage of findings. We have to decide which findings represent real risk, what deserves scarce engineering attention, and where the business should act first. If AI makes the first part exponentially cheaper without making those decisions equally cheap, the economics of the entire process begin to shift.

The Asymmetry Nobody Priced

Bugcrowd described the mechanism best: “AI changed the economics of any human-validated system. Convincing content got cheap to produce, and checking whether it’s actually correct did not.” Two costs that used to travel together came apart. Generation got cheaper by orders of magnitude. Verification held its price because verification is still a human reading carefully, reproducing a claim, and deciding whether it holds up.

Then it got stranger. As the models improved, their output became more fluent, better structured, and more confident, which made it more expensive to check than crude junk ever was. Obvious garbage takes a minute to dismiss. A polished, internally consistent, completely wrong report has to be run all the way to ground before anyone can safely close it. Each gain in the model’s fluency arrives as a larger demand on a human expert’s attention.

Where the Expensive Work Went

Here is the consequence that matters for anyone running a security program. When a finding gets produced but never verified, the verification still has to happen by whoever is qualified to make the call. And that is always the most expensive person in the building to interrupt.

In open source, it’s an unpaid maintainer with a few hours a week. For example: Of curl’s seven-person security team, a single report typically pulls three or four of them for anywhere from 30 minutes to three hours, and Daniel Stenberg has described losing the better part of an hour to one fabricated, 400-line submission that was wrong from top to bottom. Inside a company, the arithmetic is worse because the people equipped to disprove confident fiction are your senior responders and your most experienced engineers. The exact people whose time you were trying to protect when you reached for automation in the first place.

So, the efficiency that looks so clean on the production side reappears as a tax on the validation side, and it gets paid by the most senior people you have. That is the real total cost of ownership of a false positive; it travels downstream and settles on the desks that can least spare the hours.

The Cost No Dashboard Shows

There is a further cost that no triage metric captures. Stenberg has been candid that the relentless slop “takes a serious mental toll,” and he framed shutting curl’s bounty down as a way to protect his team’s “survival and intact mental health.”

That matters as a human concern, but security leaders should recognize it as an operational one as well.

The more time experienced people spend disproving plausible but incorrect findings, the less attention they have for the signals that genuinely deserve investigation. Over time, high volumes of low-confidence output can create fatigue and erode trust in the system producing it.

And that creates a dangerous possibility: the real vulnerability is there, but it is buried inside so much plausible noise that the people responsible for finding it have learned to distrust what they are seeing.

What Security Leaders Should Optimize For

This is where I think the AI conversation needs to evolve. The question cannot simply be how much more security work AI allows us to produce. We also need to ask how much of that output we can confidently act on.

In offensive security, finding a possible weakness is only the beginning. A useful result requires proving that the weakness is real, understanding how it can be exploited, determining what an attacker could actually accomplish, and translating that into a decision someone can make. AI is extremely useful across that process. It can expand coverage, accelerate analysis, identify patterns, and help practitioners investigate more possibilities than they could manually. But increasing the number of possibilities makes judgment more important, not less.

For a CISO, that distinction has practical consequences. Security programs already operate with finite engineering capacity, finite budgets, and finite attention. Every finding we escalate asks someone else in the organization to spend some portion of those resources.

So, the standard cannot simply be, “Can AI find more?” It also has to be: “What level of confidence do we require before asking the organization to act?”

That is where the economics become inseparable from accountability.

Someone Owns the Truth

The Linux kernel keeps landing on the cleanest version of all this. Linus Torvalds’ frustration has never been with AI as a tool. It is with the people who use AI to generate issues and then drop the unverified output on maintainers to sort out. The principle the kernel settled on is worth borrowing for findings of every kind: a human owns every line, whether AI helped write it or not.

Accountability does not transfer to the model.

A model can surface a candidate issue. It can analyze evidence. It can produce an exceptionally convincing explanation of why something may be vulnerable. But someone still has to decide whether that finding is real. Someone has to determine whether it matters. And ultimately, someone has to be accountable for asking the organization to act on it.

That responsibility becomes more consequential as AI makes producing plausible output cheaper and easier.

For security leaders, that may be the more important way to think about AI’s impact on our field. The technology can make us faster and give our teams tremendous leverage. But the goal was never to maximize the number of findings we produce.

The goal is to make better decisions about risk. As the cost of producing answers continues to fall, the value of being able to trust those answers will continue to rise.