Auditing the Content You Already Have Before You Point AI at It

Most content audits die in the spreadsheet. Somebody exports 1,200 URLs, adds seven columns, colour-codes forty rows, and then a campaign lands and the tab never gets opened again. That wasn’t laziness. A proper audit used to cost either three weeks of someone’s attention or £4,000 to £6,000 of freelance time, and the output was a list of recommendations nobody had capacity to act on.

That economic barrier is gone. Classification of a 1,200-URL library now costs a couple of pounds in API spend and about two afternoons of your own time. Which creates a new problem: because it’s cheap, teams are running audits as a warm-up exercise and then pointing AI at whatever the audit spat out. The audit isn’t the warm-up. It’s the thing that decides where your AI production budget goes for the next quarter, and if you classify assets badly you will spend that budget generating new posts that compete with pages you already own.

The audit is a budget decision, not an inventory

Here is the question an ai content audit exists to answer: for every asset I own, is the highest-return AI investment a refresh, a consolidation, a retirement, or nothing at all, and what’s left over for net-new production?

Ahrefs’ Content Audit and Semrush’s equivalent won’t answer that, because they sort by traffic. Traffic tells you what’s working now. It says nothing about whether a page is decaying, whether it’s eating another page’s rankings, or whether fixing it takes twenty minutes or two days of subject-matter interviews. Those three variables are what determine return. So classify on those three, in that order.

1. Decay: is it falling, flat, or was it never alive?

Pull 16 months from Google Search Console. The UI will do a period-over-period comparison, but for anything over a few hundred URLs use the bulk export to BigQuery, or searchconsole via the API into a pandas frame. You want clicks and impressions for the trailing 90 days against the same 90 days a year earlier, per URL.

Then band them:

• Cliff: down more than 40% year on year, with impressions falling too. Something changed in the SERP or the page went stale.
• Slow: down 10 to 40%. Normal ageing.
• Stable: within ±10%.
• Seasonal: down heavily but with a matching trough in the prior year. Not decay. Leave it alone.
• Never ranked: fewer than 25 clicks in 180 days and no upward impression trend. This is usually the biggest bucket and it’s the one people mistake for an opportunity.

Impressions matter as much as clicks. A page holding impressions while losing clicks has a title, snippet, or SERP-feature problem, and that’s a ten-minute fix. A page losing both has lost relevance, and that’s a rewrite.

2. Cannibalisation: which of your own pages are fighting

Two checks, and you need both. Query overlap first: for each pair of URLs, how many queries do they both appear for in GSC, and how often do their positions swap between weeks? Position instability across a shared query set is the real signal, not similarity alone.

Semantic similarity second. Embed title plus H1 plus the first 400 words of body text. You do not need a paid API for this; sentence-transformers with bge-small-en-v1.5 runs on a laptop and handles 1,200 documents in under two minutes. Flag any pair above 0.86 cosine similarity. Below about 0.80 you get noise, above 0.92 you only catch near-duplicates and miss the genuinely damaging overlaps.

Cross the two lists. A pair that’s both semantically close and swapping positions on shared queries is a consolidation candidate. Semantically close but no query overlap is usually fine, two posts for different funnel stages.

3. Update cost: the variable everyone leaves out

This is where most audits fail, because it can’t be pulled from an API. Update cost is the honest estimate of what it takes to make the page good again, and it determines whether a refresh is worth doing at all.

Four levels work:

• Cheap (under 45 minutes): dated statistics, stale year references, broken outbound links, a missing FAQ block, weak internal linking.
• Structural (2 to 4 hours): the argument is wrong or the page answers a different question than the one people search. Needs a new outline and substantial rewriting, but no new information.
• Research (a day-plus, plus somebody else’s calendar): needs fresh data, a customer quote, a product screenshot from a release you haven’t shipped, or a legal review.
• Retire: nothing to recover. Redirect or delete.

An LLM can propose this classification well if you give it the page text plus the GSC query set, and it is right often enough that you’re reviewing rather than authoring. It is systematically over-optimistic about the Research bucket, because it can’t see that your only compliance reviewer is on maternity leave. Assume you’ll bump 10 to 15% of its Cheap calls upward.

Actually running it

Crawl with Screaming Frog SEO Spider (free to 500 URLs, roughly £200 a year beyond that) with the GSC and GA4 APIs connected in Configuration → API Access, and Ahrefs connected if you have it. Export internal_all.csv. You now have URL, title, meta, word count, indexability, inlinks, clicks, impressions and referring domains in one file.

Join your GSC year-on-year frame to that, add the similarity pairs, and you have the input rows. Then classify in two passes with the Message Batches API, which runs asynchronously at 50% of list price.

Pass one, everything. Claude Haiku 4.5 (claude-haiku-4-5, $1 per million input tokens, $5 output) with structured outputs and hard enums. Roughly 2,500 input tokens per URL (title, meta, first 600 words, top 15 queries with positions, the decay band you computed) and about 200 out. For 1,240 URLs: 3.1M input, 248K output, $4.34 at list, $2.17 via Batches.

Pass two, the shortlist. For the 60-odd assets that carry real traffic and show decay, send the full page text to Claude Opus 5 (claude-opus-5, $5/$25) and ask for a specific refresh brief: which sections are wrong, which queries the page is failing to answer, what to cut. About 12,000 in and 1,200 out each, so $5.49 list, $2.75 batched.

Total API spend, both passes: under $5, about £4. Compute the decay bands and similarity in code, never in the model, and pin every classification to an enum so the output pivots cleanly.

decay_class    cannibal_flag   update_cost   urls   clicks_180d  clicks_prior
cliff          cluster_head    cheap           61       14,208       38,940
cliff          duplicate       cheap           23          806        4,115
slow           none            cheap           88       22,540       26,102
slow           duplicate       structural      34        1,977        3,050
stable         cluster_head    cheap          214       61,330       59,870
seasonal       none            cheap           71        9,442        9,120
never_ranked   duplicate       retire         247           91          104
never_ranked   none            research       165          310          288
(11 more rows)                               337       51,300       46,416
TOTAL                                       1,240      162,004      168,905

Look at row four and row seven together. Two hundred and forty-seven URLs producing 91 clicks in six months, all of them duplicating intent with something else in the library. That is four years of somebody’s content calendar, and it is actively suppressing the pages you want to rank.

What the classification tells you to fund

The matrix is boring and that’s the point:

From the table above, that resolved to 61 AI-assisted refreshes, 47 consolidations and redirects, 19 briefs for genuinely uncovered queries, and permission to ignore roughly 900 URLs entirely. Net-new production dropped from eight posts a month to three, and the refresh cohort recovered about 9,400 monthly clicks over the following quarter, at a fraction of what eight new posts a month was costing. Refreshing a page that already has links and history is the cheapest ranking you will ever buy, and it is the work AI is genuinely good at: given the existing page, the failing queries and a clear brief, drafting the revision is a constrained task rather than an invention.

Feed the output straight into how you plan the quarter rather than treating it as a separate artefact; the mechanics of that handoff are covered in AI-assisted content strategy and planning.

The four ways this goes wrong

Trusting the model’s read of the page. It will tell you a post is comprehensive because it’s long. Always give it the query data alongside the text, and always ask what the page fails to answer rather than how good it is.

Confusing a SERP change with decay. If a page lost 60% of clicks in one week in March, that’s an update or an AI Overview, not staleness. Check whether impressions held. Rewriting won’t help if the click is being intercepted above you.

Consolidating without checking backlinks. Run the redirect list through Ahrefs or Majestic first. A dead 400-word post with eleven referring domains is a redirect target, not a deletion.

Running it once. Decay bands go stale in about a quarter. The whole pipeline, once built, costs a few pounds and an hour to re-run, so put it in the calendar for the first week of every quarter and let the previous run’s classifications be the diff.

If you want a test that fits in an afternoon: take the 200 URLs with the most clicks, run just the decay banding and the year-on-year comparison, and count how many are in Cliff. Most teams find eight to fifteen pages that were quietly losing a third of their traffic while the team shipped new posts next to them.