FinOps for AI: unit economics and cost control for LLM apps
What the FinOps Foundation says about managing AI spend, the metrics that matter for LLM applications, how to allocate and budget token costs, and the levers that actually lower the bill.
Last updated October 22, 2026
What is FinOps for AI?
FinOps for AI applies FinOps practices to the cost of AI services: making token, API and GPU spend visible, allocating it to the teams and products that cause it, budgeting and forecasting it, and lowering it without hurting quality. For LLM apps, it means knowing the cost of each request, user and feature.
What does the FinOps Foundation say about FinOps for AI?
The FinOps Foundation, which maintains the FinOps Framework, publishes a FinOps for AI topic hub and a set of working-group papers. Its FinOps for AI Overview (last updated February 17, 2026) makes a few points worth knowing:
- The basic cost model is still price times quantity: you lower costs by paying a lower rate or using less.
- AI pricing is harder to manage than traditional cloud: many model variants and frequently changing SKUs, token metering where billed tokens can differ from what users submit, and GPU capacity that is scarce and volatile.
- AI spend is now driven by groups outside engineering, such as product, marketing and sales, and model choice is itself a cost decision, because a smaller model may meet the requirement for less.
- Its cost drivers include input and output tokens, API calls, GPU compute time, training, storage, data transfer and commitments.
The Foundation’s topic page also stresses value, not only cost: tying AI spend to business outcomes such as productivity, revenue and customer experience, and checking that each initiative earns its place.
Which metrics should you track for LLM costs?
Unit economics means expressing spend per unit of something your business cares about. The first four rows below are KPIs the FinOps for AI Overview names; the rest are common ways to apply them to an LLM application.
| Metric | How to calculate it | What it tells you |
|---|---|---|
| Cost per inference (request) | Input tokens × input price + output tokens × output price, per request | The base unit everything else is built from |
| Cost per token | Total spend ÷ total tokens, split by input, output and cached input | Whether your mix of models and caching is getting cheaper |
| Cost per API call | Spend ÷ calls, including retries and tool calls | Hidden cost from retries and agent loops |
| Anomaly detection rate | Share of cost spikes caught by alerts before the invoice | Whether budgets and alerts are working |
| Cost per active user | Monthly LLM spend for a product ÷ its monthly active users | Whether the feature can be priced profitably |
| Cost per feature | Spend tagged to a feature ÷ uses of that feature | Which features drive spend |
| Cost per customer | Spend tagged to a customer or tenant | Margin per account; who needs a usage limit |
| Cache hit rate | Requests (or input tokens) served from cache ÷ total | How much repeated work you avoid paying for |
| Model mix | Share of requests and spend per model | Whether cheap models handle the simple work |
How do you allocate and tag LLM costs?
The FinOps for AI Overview recommends tagging AI resources by project, environment, workload type (training or inference), team, cost center and usage type, and starting with showback (showing teams their costs) before moving to chargeback (billing them).
LLM API calls are not cloud resources you can tag in a console, so the tags have to travel with each request. In practice that means:
- Separate API keys or projects per team or environment, so provider billing splits at least that far.
- Request metadata for feature, customer and user, captured by a gateway or your logging layer.
- A price table per model, kept current, so tokens can be converted to dollars per request.
- Shared costs, such as a common system prompt or evaluation runs, split by an agreed rule.
How should you budget and forecast AI spend?
The Foundation’s guidance is to build AI budgets bottom-up by component, revise them more often than cloud budgets, give pilot projects flexible budgets, and forecast more frequently because AI costs are volatile, especially in early phases. It also lists usage quotas, throttling and anomaly alerts among its optimization levers.
For an LLM application, a working setup usually has three layers:
- Provider limits as the last line of defense: for example, Anthropic lets you set your own spend limit below your tier’s cap, and OpenAI documents hard spend limits that return errors once reached.
- Budgets per team, key or customer enforced before requests reach the provider, so one runaway workload cannot exhaust the shared limit.
- Alerts at thresholds below each budget, and on sudden changes in daily spend or tokens per request.
A simple forecast multiplies expected requests by average tokens per request and by price per token, per model. The LLM cost calculator does this for you.
What are the main levers for lowering LLM costs?
The FinOps Foundation paper Optimizing GenAI Usage lists model selection, prompt routing, caching, batching, token optimization and more. The levers that apply to most API users:
| Lever | What it means | Trade-off |
|---|---|---|
| Model choice | Use the smallest model that meets the quality bar; route simple requests to cheaper models | Needs evaluation so quality does not silently drop |
| Prompt length | Trim system prompts and context; cap output length | Too little context can lower answer quality |
| Provider prompt caching | Reuse a repeated prompt prefix at a discounted input rate. See OpenAI and Anthropic docs | Only helps when the start of the prompt repeats exactly; cache writes can cost more than normal input |
| Response caching | Return a stored answer for repeated or similar requests | Stale or wrong answers if the cache is not scoped carefully |
| Batching | Send work that can wait to a batch API. OpenAI’s Batch API and Anthropic’s Message Batches API each list a 50% discount | Results are not immediate; OpenAI states a 24-hour completion window |
| Budgets and quotas | Hard and soft limits per team, key or customer | Limits that are too tight block real users |
Prices checked October 22, 2026; providers change prices often — confirm on the provider’s page. For per-token prices, see the LLM API pricing guides.
Who is responsible for AI costs?
The FinOps for AI Overview lists a wide cast: data scientists, data engineers, software engineers (including prompt engineers), business analysts, DevOps engineers, product managers, leadership and end users, alongside finance, procurement and FinOps teams. For an LLM application the split often looks like this:
- Engineers choose models, write prompts and set up caching and routing, so they control most of the unit cost.
- Product managers own cost per user and per feature, and decide where a cheaper model is good enough.
- Finance and FinOps own budgets, forecasts, allocation rules and commitments with providers.
- Leadership decides which AI initiatives are worth their cost.
Cost is only half the picture: a cheaper model that gives worse answers is not a saving. Tracing and evaluation tools such as Perimattic AI Suite help you watch quality alongside spend.
How does Perimattic LLM Gateway help with FinOps for AI?
Perimattic LLM Gateway covers the request-level work that FinOps for AI depends on:
- Cost tracking and attribution by team, feature and customer, with alerts.
- Budgets and rate limits per team, API key or customer.
- Routing rules that send each request to a model by cost, latency or task.
- Exact-match and semantic caching of responses.
Get started: Sign up and our team will get in touch, or estimate your own savings with the LLM gateway savings calculator.
Related guides and tools
FinOps for AI, answered
What is the difference between FinOps for AI and AI for FinOps?
FinOps for AI is managing the cost and value of AI services you use. AI for FinOps is using AI to help do FinOps work, such as analyzing bills or writing queries. The FinOps Foundation treats them as separate topics; its FinOps for AI overview paper says using AI to perform FinOps is out of scope.
How do I allocate LLM costs to teams or customers?
Provider invoices usually show spend per organization, project or API key, not per feature or customer. To allocate further, record the team, feature and customer on every request, either with separate keys or with metadata your gateway or logging layer captures, then multiply each request’s tokens by the model’s price.
Why are AI costs harder to forecast than cloud costs?
Usage is metered in tokens that vary with every prompt and response, model prices and versions change often, and agentic workflows can make a variable number of calls per task. The FinOps Foundation recommends revising AI forecasts and budgets more often than for traditional cloud spend, especially early on.
Want one API for every model, with costs under control?
Sign up to tell us which models and providers you use, and our team will get in touch to set up the LLM Gateway for your stack.
Sign up