AI cost audit: how to find where your company is burning budget.

The AI invoice is growing. Nobody knows why. Most companies I talk to know exactly how much they paid OpenAI last month — and have no idea what a single closed ticket, a single generated document, or a single user session with the in-product AI assistant actually cost them.

This isn't an accounting bug. It's a missing layer that nobody in the org has needed until now. Cloud had FinOps. AWS Cost Explorer. Resource tagging. Showback / chargeback. AI doesn't have any of that standardised yet — and costs are growing double-digit every quarter.

This piece shows how to run a quick AI cost audit in 3–5 days yourself. If it surfaces something bigger than “I have two expensive models doing refactoring” — that's a good starting point for a full audit. If it shows everything is under control — that's not 3 wasted days, that's a good night's sleep.

Three numbers that reveal the problem in 30 minutes

Before you dive into details, check three numbers. Any company can calculate them in half an hour, with no tooling.

1. Cost per useful output. Not the API line on your invoice — the cost of one ticket that actually got closed, one document somebody actually read, one summary somebody actually used. Many companies measure “cost per request” — but 40% of requests are retries, errors and replies nobody used.
2. AI cost as % of gross margin. Below 3% you sleep well. Above 10% you have a margin problem, not a cost problem. Between 3% and 10% is the zone where an audit pays back fastest — because the sources are concrete and findable.
3. Power-user concentration. The top 5% of users typically consume 30–50% of an AI feature's cost. If you don't know that percentage for your product, you're blind to the biggest unit-economics risk.

Step 1 — Break down the bill

The aggregate invoice from OpenAI / Anthropic / Azure is an abstraction. First move is to split it into four views: per provider, per model, per use-case, per department. All four, not just one.

Most billing systems show the first view. The other three you have to reconstruct — from application logs, request metadata, project tagging on your side. If you're not tagging requests, that's point zero of the first audit: tagging.

A typical breakdown looks roughly like this (example from a SaaS with $9.5k / month in AI spend):

First observation: two expensive models (gpt-4o, claude-opus) account for 56% of cost. Second: support bot uses gpt-4o for a task that gpt-4o-mini handles at 1/15 the price. Third: there's subscription waste — 3 inactive Copilot seats are ~$190/mo burning on empty accounts.

Step 2 — Expensive model on a simple task

The most common and cheapest-to-implement source of savings. Most teams start a project with “drop in GPT-4o, ship it” — and it never gets revisited. A year later, the support bot handling 80% of “how do I reset my password” questions is still running on the most expensive model on the market.

The practical rule is simple: intent-based model routing. A cheap first model (e.g. gpt-4o-mini) classifies whether the task needs a reasoning-class model or a fast/cheap one will do. The rest of the request goes down the right lane.

The pseudocode we ship most often:

async function route(userMessage) {
  // 1. Cheap router picks the lane
  const intent = await classify({
    model: "gpt-4o-mini",
    messages: [{role: "system", content: ROUTING_PROMPT},
               {role: "user", content: userMessage}],
    // max_tokens 4 — reply is one of: SIMPLE / COMPLEX / ESCALATE
    max_tokens: 4,
  });

  // 2. Route to the right model
  if (intent === "SIMPLE")   return ask("gpt-4o-mini",   userMessage);
  if (intent === "COMPLEX")  return ask("claude-sonnet", userMessage);
  if (intent === "ESCALATE") return handoffToHuman(userMessage);
}

What you get: cost on the expensive model drops 60–80%, output quality measured by human review doesn't drop (and often goes up — because gpt-4o-mini doesn't start “creatively” answering simple questions).

Step 3 — Cost per useful output

The hardest but most important number in the whole audit. Definition: the cost of one AI action that actually generated business value.

“Business value” is not “request returned HTTP 200”. It's: a ticket closed positively, a document was read, a summary was copied, code was accepted in a PR. Most companies measure requests. They should be measuring useful outputs.

We had a client with a beautiful dashboard: $0.0085 / request. Looked great. After we added a “did the user accept the answer” layer, it turned out 54% of requests were retries of the same intent. Real cost per useful output: $0.0185. More than twice as much.

How to measure it:

• Tag every request with metadata: user_id, session_id, intent, outcome.
• In the application, log an “outcome” signal: user clicked “helpful” / “copied” / “close ticket” / “send” — whatever “I used this” means in your process.
• In the cost layer: sum the cost of requests belonging to sessions with a positive outcome — those are the “useful” ones. The rest is overhead.

Tools that do this out of the box: Langfuse, Helicone, LangSmith, Portkey. The choice depends on your stack — but rolling any of them out is cheaper than measuring it by hand after the fact.

Quick wins that almost always work

A list of recommendations that land in the top tier in 80% of audits. Low implementation effort, high financial impact:

The first four combined typically deliver 50–70% API cost reduction in the first 30 days after the audit. With no quality loss — because all four are pure technical optimisation, not feature reduction.

What you won't find doing this alone

The steps above get you to about 70% of a full audit's value. The remaining 30% is the parts you can't do without an outside perspective:

• AI feature pricing. Whether the current plan covers the variable cost of power-users. That requires unit-economics modeling, not just API cost.
• Shadow AI map. Inventory of tools bought alongside the main stack by different departments — typically 30–50% of all AI subscriptions in the organisation.
• ROI per use-case. A keep / kill / iterate decision for every AI feature in the product. Requires comparing cost against measurable impact on retention, conversion or revenue.
• Implementation roadmap. A concrete plan of what to ship in 30/60/90 days — with priorities, dependencies and task owners.

Wrap-up

An AI cost audit isn't complicated. It's a discipline — doing three things most teams don't: breaking down the bill, computing cost per useful output, and checking power-user concentration.

Do it yourself today, and within a week you'll know whether you have a $1,200/year problem or a $50,000/year problem. That's enough to make an informed call: stay DIY, or bring in someone to run the full audit with deliverables and a roadmap.

— Andrzej Datta, floomi. Questions, comments, your own cases: hello@usefloomi.com.