The question comes up in sales calls, and it deserves a straight answer. Frontier models are good at security work, and we build Konvu on top of them. My job is to evaluate how they perform on vulnerability triage, so I spend my days measuring where they hold up and where they stop.
The short answer: the model is one component. Four things it can't give you turn out to be most of the product.
Most of what a frontier model knows is dead weight here
Claude, Codex, pick your favorite. They write sonnets, Kubernetes manifests, and legal memos with equal ease. The question we answer all day needs almost none of that breadth: is this vulnerability actually exploitable in this environment?
The question is narrower than it looks. A CVE with a scary severity score might sit in a dependency that never ships in your runtime artifact, behind input an attacker can never touch.
Breadth isn't useless. Broad pretraining is why an LLM can read code and reason about security at all, and a specialized system builds on that foundation instead of replacing it. But breadth doesn't answer the narrow question. Narrow and deep beats wide and brilliant here, and depth has to be built on purpose.
The exploitability data was never written down
Exploitability comes down to a set of conditions. Which function has to be called, with what input, under which configuration, in what kind of deployment. For a ReDoS, the conditions include an attacker controlling the string being matched and a regex engine that actually backtracks; run the same pattern on a linear-time engine and there is nothing to exploit. For unsafe deserialization, untrusted bytes hitting the sink is only the start. You also need a usable gadget chain on the classpath.
That knowledge mostly doesn't exist as data. Fragments sit in advisories, proof-of-concept repos, and conference writeups. For most of the vulnerabilities we investigate, nobody ever wrote the conditions down at all. A model can't have learned what nobody recorded.
So we record it. We maintain a vulnerability database enriched with exploitability conditions, built vulnerability by vulnerability. Our agents check those conditions against your code and your environment instead of extrapolating from whatever the training data happened to contain. It's slow, unglamorous work, and the part I'd least want to compete against.
Models declare victory too early
Anyone who codes with an LLM knows the moment. The model announces the task is done, and it isn't. In a chat window this costs you an eye-roll and a follow-up prompt.
In triage the same behavior is dangerous, because the failure is silent. A model that outputs "not exploitable" after an incomplete investigation has produced a false negative that looks identical to a clean verdict. Nobody pushes back, because nobody knows there was more to check.
A human babysitting every conversation catches this, and a product can't ship with a babysitter. So the supervision has to live in the system: bounded tasks instead of free-form exploration, and verdicts that carry their evidence (the call path, the configuration, the conditions that held). I made a version of this argument in my RAISE Summit recap: the less your product depends on an agent's open-ended reasoning, the more viable it is.
Trained on approval, graded on exploits
The deeper problem is the objective. Model providers publish part of their training recipe, and optimizing for human preference is in it. Models learn, in part, to produce answers people rate highly, and research on sycophancy shows the drift that follows. Labs also train on verifiable outcomes like passing tests, and they work against sycophancy directly. None of those objectives grades whether a vulnerability is exploitable in your environment.
An exploitability verdict is right or wrong, whether anyone liked reading it. So the question that matters for any AI security product is what the system is graded against.
We grade against ground truth. Our evaluation corpus has two kinds of data. Isolated cases work like unit tests: a piece of code, a weakness class, a known answer. The valuable part is full open-source repositories with known vulnerabilities, labeled by our own security engineers, where the verdict depends on context the way it does in production.
Before we trust any LLM judge to grade at scale, we calibrate it against those expert human labels. A judge that agrees with our engineers on the cases we labeled earns the right to grade the ones we haven't. When it disagrees, a human reviews the disagreement, and the corpus grows by one more case.
Labeling is expensive precisely because the knowledge is rare. Every labeled case sharpens the next evaluation, and the corpus stays useful across model releases.
Takeaway: ask what the system is graded against
Frontier models keep improving, and we benefit every time they do. The exploitability conditions nobody wrote down and the ground truth to grade verdicts against don't ship with a new release. Someone has to build them.
So when a vendor says their AI triages vulnerabilities, ask them why you couldn't just use Claude. The answer should sound like work.