Which LLM Is Best for Content Audits at Scale? I Tested 6 OpenAI Models on 100 Pages

TL;DR

I wanted to answer one question: if you hand the same content audit to six different OpenAI models, which one finds the most real problems for the least money, without making things up? So I ran the test. Same 100 pages, same brief, six models, and every single finding checked against the page it came from.

• Best value: GPT-5.4. It found 717 real issues at 4.2 cents per page with a 3.4% hallucination rate. The best mix of recall, accuracy and price.

• Highest recall: GPT-5.6 Sol. It caught the most real issues (767) but cost 18.1 cents per page, more than 4x GPT-5.4.

• Cleanest: GPT-5.5. It hallucinated 0.0% of its findings, but at 20.4 cents per page it is the priciest way to audit.

• Avoid for volume: GPT-5.4 mini and GPT-5.6 Luna. Cheap and fast, but they invented 16.9% and 19.4% of what they reported.

• The whole run: 100 pages, 6 models, 2,717 findings pooled, 53.33 dollars in total spend.

Why I ran a six-model content audit benchmark

Content audits do not scale by hand. I have spent 14 years in SEO and the last two deep in GEO, and the biggest bottleneck I keep hitting is the one every content team hits: reading every page closely enough to catch what is actually wrong. On a 40-page site you can do it. On a 4,000-page site you cannot, so most audits quietly collapse into crawler reports: broken links, missing titles, thin-content flags. A crawler can tell you a page is 300 words. It cannot tell you the page never answers its own question, or that it contradicts itself three paragraphs apart. That gap is exactly where rankings and AI citations are won and lost.

Large language models read for meaning, which makes them the obvious tool for the job. But two worries stop most people trusting them at scale: which model is actually good at this, and how much of what it reports is invented? I did not want a hunch, so I treated it like an experiment. I built a controlled test, ran six models through the exact same task, and scored every finding they produced. This is the kind of AI SEO automation work I now lean on for clients, and the results surprised me in a few places.

How the test worked, in plain English

The setup is simple to describe. I took 100 pages, a deliberate mix of transactional pages (product, pricing, checkout) and informational pages (guides, FAQs, support articles). I gave all six models the identical brief: read each page, report every content problem you find, and for each one, quote the exact text on the page that proves it.

Then I did the part most people skip. Instead of trusting any single model, I pooled every finding from all six into one master list, and checked each finding against the page it came from. If a model claimed a page contradicted itself, the quoted evidence had to actually exist on that page. Findings that passed became real findings. Findings whose evidence could not be verified were marked as hallucinations. This pool-and-adjudicate design is borrowed from how search engines are scored in academic information retrieval (the TREC pooling method), and it is the only honest way to measure recall and accuracy at the same time.

Three numbers do most of the work in this study:

• Real findings. Problems a model reported whose quoted evidence checked out. My proxy for recall, or how much a model catches.

• Hallucination rate. The share of a model’s findings where the evidence was fabricated or could not be verified. My proxy for how much it makes up.

• Cost per page. Total API spend for that model divided by 100 pages, using published OpenAI API pricing.

One caveat before the numbers: these figures come from a controlled run and are scaled proportionally. Treat them as a snapshot of these specific model versions, not a permanent ranking. Model quality shifts with every release, so the method matters more than any single row in the table.

I have kept this part short on purpose. If you want the full mechanics, exactly how I scored hallucination, where the pooling maths comes from, and where the numbers stop being reliable, it is all written out in the methodology, hallucination, and limitations sections near the end.

The leaderboard: six models, one value winner

Sorted by real findings kept, here is the full board. GPT-5.6 Sol caught the most problems. Then look at the cost column, and look at what GPT-5.4 did right next to it.

GPT-5.4 is the one I would put into production. It found 717 real issues, within 7% of the top model, at 4.2 cents a page. That works out to 0.58 cents per real finding, more than 4x cheaper than Sol and nearly 7x cheaper than GPT-5.5. It hallucinated only 3.4% of its findings, low enough to trust with a light human review. And 79% of its findings were unique to it, meaning it was not simply repeating what the cheaper models had already flagged. High recall, low noise, low price, and it pulls its own weight in the pool. That is the whole scorecard.

Findings versus cost: the value quadrant

The leaderboard hides one thing: the trade-off between how much a model catches and what it charges you to catch it. Plot those two against each other and the decision gets obvious.

The sweet spot is the top-left corner: lots of real findings for very little money. GPT-5.4 sits alone in it. The two reasoning models, Sol and GPT-5.5, deliver excellent accuracy but live far to the right, four to five times the cost per page. The two amber bubbles, mini and Luna, look cheap until you remember they invented roughly one in six of their findings, so their raw counts are inflated by noise. For a one-off audit of a handful of critical pages, a reasoning model’s premium can be worth it. For auditing thousands of pages on a schedule, it is the difference between a project you can afford and one you cannot.

Signal versus noise: hallucination, agreement and severity

Raw finding counts lie. A model that reports a thousand issues looks impressive until you learn a fifth of them are made up. Three checks separate the signal from the noise.

Hallucination: how much each model makes up

Every finding had to prove itself with a quote from the page. Here is how often each model’s evidence failed that check, sorted from cleanest to noisiest.

The split is stark. The reasoning models and GPT-5.4 all stayed under 5%. The two cheapest fast models, mini and Luna, invented close to one in five of their findings. That is the trap with the cheapest models: the price per finding looks unbeatable until you realise you are paying a human to delete the fabrications. Past roughly 15% hallucination, the noise costs you more than the model saves.

Agreement: how many models found the same thing

I also tracked how often models agreed. When several independent models flag the same problem, you can trust it without re-checking. When only one does, you have to look.

85% of all real findings came from a single model. That sounds alarming, but it is the entire reason a benchmark like this matters: no single model sees everything. Run only one model and you would have missed most of what the others caught. Pooling several models and then adjudicating is what turns a pile of maybe-issues into a ranked, trustworthy fix-list. It is also why I never rely on one model for a client audit that matters.

Severity: how bad were the problems

Not every issue deserves your time. I graded each real finding by how much it could hurt rankings, AI citations or conversions.

1,491 findings, more than half the total, were high or critical. These are not cosmetic. They are the contradictions, coverage gaps and wrong-entity errors that make a page lose a ranking or get skipped by an AI answer engine. That is the payoff of auditing for meaning instead of crawling for structure.

What 100 pages actually got wrong

Across all six models, the pooled findings fell into ten recurring problems. Two of them dominated everything else.

Coverage gaps and internal contradictions were 60% of everything found. Neither shows up in a crawler. A crawler cannot tell that a page never answers its own headline question, or that the shipping time in the hero contradicts the shipping time in the FAQ. Those are precisely the errors that AI answer engines punish, because a model reading your page for a citation hits the contradiction and moves on to a cleaner source. If you want the deeper version of why this matters for AI search, I wrote it up in my guide to GEO.

The cost math at scale

Here is why the value winner matters. Four cents a page sounds trivial at 100 pages. Multiply it across a real site and the models separate hard.

At enterprise scale, the model choice becomes a four-figure decision every time you re-audit. GPT-5.4 gets you 94% of the top model’s recall for less than a quarter of the price. That is the definition of the right tool for volume work. If you are auditing a small, high-stakes set of pages once, a reasoning model’s near-zero hallucination can justify the premium. For everything else, value wins.

How to run this audit yourself

You do not need my exact setup to get most of the benefit. The workflow is what matters, and it is repeatable:

• Use a value model for the first pass. A GPT-5.4-class model reads every page and reports findings with quoted evidence, cheaply enough to run across the whole site.

• Always demand evidence. The single most important instruction is "quote the exact text that proves each finding." It is what lets you catch hallucinations automatically instead of by hand.

• Pool, then adjudicate. For high-value pages, run more than one model, pool the findings, and verify each quote against the page. Agreement between models is your confidence signal.

• Rank by severity, not volume. A list of 2,700 issues is noise. A list of 83 critical and 1,408 high issues, sorted, is a roadmap.

• Re-run on a schedule. Content decays. Contradictions and stale claims creep back in. An audit you can afford to re-run monthly beats a perfect one you run once a year.

This is the workflow I now run for clients auditing content at scale, and it is a large part of what AI has actually changed about SEO. The bottleneck was never really the writing. It was the reading. The teams that win in AI search are the ones who can check 10,000 pages for meaning as cheaply as they used to check them for broken links. If you want this run on your site and turned into a ranked fix-list, that is exactly what I do under AI search consulting and GEO consulting.

How I ran it: methodology and technical nuances

The short version above is enough to trust the results. This is the long version, for anyone who wants to run the same test or poke holes in mine. The pipeline has four parts: a deterministic pre-check that every model shares, a single model pass that produces findings with quoted evidence, a pooling step that merges those findings into distinct claims, and one human pass that decides what is real. Each part exists to remove a specific way the numbers could lie.

Pool and adjudicate: one pass, two outputs

Every model reviewed the same page and returned structured findings. I pooled all of those findings into distinct claims, keyed by page, section, issue category, and the normalized evidence quote, so the same problem spotted by three models collapses into one claim. Then a human accepted or rejected each distinct claim exactly once. That single accepted set does two jobs at once: it is the ground-truth key I score every model against, and it is the actual fix-list for the site. Scoring six models the naive way would mean re-reading every page six times over. Pooling first, then judging once, is the shortcut, and it is the same pooling method academic search evaluations have used for decades to score retrieval systems without re-reading the entire web.

Fair retrieval: every model got the same head start

Before any model saw a page, that page went through a deterministic pipeline: fetch and render, extract the readable content, then run a fixed set of mechanical checkers. The output of those checkers was handed to every model as known findings. This matters for fairness. It stops a model padding its score with the easy mechanical stuff a script already caught, so the benchmark measures the judgement a model adds on top, not its luck at re-spotting a broken link. The extracted page content was always treated as data, never as instructions, which is the basic guard against prompt injection when you feed scraped pages into a model.

What is measured, and what is still provisional

Three things are measured directly and I stand behind them: real findings kept after the evidence check, the hallucination rate, and cost metered from provider-reported tokens. Two things are provisional and I want to be clear about it: precision and recall in the strict academic sense. A single human accept-or-reject pass is a strong ground truth, but it is not the same as a full multi-rater adjudication with a second reviewer resolving disagreements. Read the recall numbers as a well-controlled proxy, not a laboratory-grade measurement.

Throughput: why tokens, not workers, set the ceiling

When you scale this to thousands of pages, the bottleneck is not how many jobs you run in parallel. Each page costs roughly 14,000 tokens to audit, and every provider caps you on tokens per minute, not pages per minute. So the rate limit becomes the real ceiling: adding more workers past that point just makes them queue. Planning a large audit is therefore a token-budget exercise, which is another quiet reason the cheaper, lighter models are attractive for volume work.

How I measured hallucination

Hallucination in this study has one precise meaning: a finding whose cited evidence cannot be found on the page. I call it the grounding drop-rate. There is no vibe check and no manual skim here. Every single finding had to pass a mechanical test before it was allowed to count.

The evidence check, step by step

The brief forced every model to quote the exact text on the page that proves each finding. That quote is what makes the check possible. Here is what happened to each finding:

• Quote required. A finding with no supporting quote was rejected outright. No evidence, no finding.

• Normalize both sides. I stripped whitespace and casing differences from the quote and from the page’s extracted text, so a real match is never missed on a stray space.

• Locate the quote. I searched the page text for the quote. If it was there, the finding was grounded and kept. If it was not, the model had invented or distorted its evidence, and the finding was dropped as a hallucination before it ever counted.

This check is deliberately strict in one direction and deliberately humble in another. It is strict because it catches both wholesale fabrications and the sneakier case where a model lightly rewrites a quote to fit its argument. It is humble because it only proves the evidence exists, not that the model’s conclusion about that evidence is objectively correct. That second limit matters enough that I spell it out in the limitations below.

The evidence-check funnel

Run every model’s output through that check and you get a clean funnel from raw findings to real ones. The gap between emitted and kept is the noise each model added.

The six models emitted 3,567 findings between them. 258 failed the evidence check and were dropped. That leaves 3,309 grounded findings, which still contain duplicates because several models often catch the same problem. De-duplicating those into distinct claims removes 592 overlaps and leaves the 2,717 unique issues the rest of this study is built on.

Limitations and honesty

A benchmark is only as trustworthy as the caveats attached to it. Here is where the edges of this one are, so you can weigh the numbers properly rather than take them on faith.

• The figures are scaled and rounded. This was a controlled run, scaled proportionally to a clean 100-page basis and rounded to whole findings. A given row can sit a fraction of a percent off its underlying rate.

• Grounding is not fact-checking. The evidence check proves a quoted line exists on the page. It does not prove the model’s interpretation of that line is objectively right. A model can quote real text and still reason poorly about it.

• Precision and recall are provisional. One human accept-or-reject pass is a solid ground truth, not a multi-rater gold standard. Treat recall as a strong proxy.

• It is a snapshot. These are specific model versions from one week of testing. Model quality shifts with every release, so the ranking will age faster than the method.

• The page mix is mine, not yours. 100 pages weighted toward transactional and informational content. Your site’s error profile, and therefore the model that suits it, may differ.

• One model, run alone, misses most of the list. 85% of real findings came from a single model, so the pooling is not a nicety, it is the whole point. Do not read any single model’s recall as complete coverage.

• Use it as a decision aid. Pick a starting model and a workflow from this, then validate on a sample of your own pages before you trust it at scale.

Appendix: definitions and the full funnel

For readers who want the raw scaffolding, here are the exact terms and the complete tables behind every chart above.

Metric definitions

The full funnel, model by model

Agreement in full

How many of the six models independently caught each distinct finding. This is the full version of the agreement summary further up.

Add the top four rows, every finding that three or more models independently agreed on, and you get the 125 highest-confidence issues, the ones worth fixing first. The long tail of 2,300 single-model findings is exactly why running one model alone is a mistake, and why pooling several is worth the extra spend on the pages that matter.

FAQ

Which LLM is best for content audits at scale?

In this benchmark, GPT-5.4 gave the best value: 717 real findings at 4.2 cents per page with a 3.4% hallucination rate. It caught 94% of what the top reasoning model caught for less than a quarter of the cost, which makes it the most cost-effective choice for auditing large sites.

Which model hallucinated the least?

GPT-5.5 hallucinated 0.0% of its findings, followed by GPT-5.6 Sol at 1.1% and GPT-5.4 at 3.4%. The cheapest fast models, GPT-5.4 mini and GPT-5.6 Luna, were the worst at 16.9% and 19.4%.

Do reasoning models find more issues than non-reasoning models?

Sometimes, but not reliably. The top-recall model here was a reasoning model (GPT-5.6 Sol, 767 findings), but second place went to a fast non-reasoning model (GPT-5.4, 717). Reasoning helped accuracy more than raw recall, and it came at four to five times the cost per page.

How do you stop an LLM from making up audit findings?

Require evidence. Tell the model to quote the exact text from the page that proves each finding, then check every quote against the page automatically. Findings whose evidence does not exist are hallucinations and get dropped. This single step is what makes an LLM audit trustworthy.

Why not just use a crawler for a content audit?

A crawler checks structure: titles, links, status codes, word counts. It cannot read for meaning. The two biggest problems in this study, coverage gaps and internal contradictions, were 60% of all findings and are invisible to a crawler. They are also exactly what AI answer engines penalise.

How much did the whole benchmark cost?

53.33 dollars in total API spend to pool 2,717 findings across six models on 100 pages. The cheapest single model ran the full 100 pages for about 1.50 dollars; the most expensive for about 20.40 dollars.

What are the limitations of this benchmark?

The figures are scaled from a controlled run and rounded, so treat them as a snapshot of these model versions, not a permanent ranking. The evidence check proves a quote exists on the page, not that the model’s conclusion is objectively correct, so precision and recall stay provisional. The page mix is my own, and 85% of findings came from a single model, which is exactly why pooling several models matters. Use it as a decision aid, then validate on your own pages.

Audit your content at scale, with data behind every finding

Looking to audit your content at scale? That is what I do. I build AI-powered audits and automations for SEO and AI search, run them across thousands of pages, and turn the raw output into a ranked, data-backed fix-list instead of another crawler report. Every finding is checked against the page it came from, so you get signal, not noise.

If you want an audit like this on your own site, or help getting cited in AI search, I offer AI search consulting and GEO consulting. Contact me and tell me what you are working on.

Related reading

• What is GEO? Generative Engine Optimisation explained

• AI SEO automation: where it actually helps

• AI Search Consulting

• GEO Consulting

• Book a strategy call