LLM gateway savings calculator
Estimate how much response caching and routing to cheaper models could cut from your monthly LLM spend. Enter your own bill, request volume and repeat rate; no vendor prices are built in.
Last updated October 22, 2026
How much could caching and model routing reduce your LLM costs?
Caching saves on the requests a cache answers: your repeat share times the hit rate. Routing saves on requests moved to a cheaper model: routable share times the price gap. This free calculator applies caching first, then routing to what remains, and shows monthly and yearly savings from your own numbers, before any gateway costs.
How much would caching and routing save you?
Enter your current monthly bill and request volume, then how many requests repeat and how many could go to a cheaper model. Every figure is an estimate before gateway costs, from your own numbers; no vendor prices are built in. To find the price gap between two models, use the token calculator or the LLM cost calculator. Filling in the calculator is free; a one-time free sign-up shows your result. What you enter stays in your browser; only the sign-up form is sent.
The starting values are example values — replace with your own. They are not benchmarks or industry figures.
Your total model API bill for a typical month.
LLM API calls your application makes in the same month.
The share of requests asking something already asked. Measure it from your logs.
Of those repeat requests, how many the cache would actually answer.
Tasks a smaller model handles well enough, judged by your own tests.
Compare the two models’ prices for your typical prompt and answer sizes.
Share of spend that fallbacks would avoid. Leave at 0 to skip this step.
Sign up once to see your results. It's free, and the results open here as soon as the form is sent.
Show resultsEstimates before gateway costs, from your own numbers. No vendor prices are built in. Assumes every request costs the average (spend ÷ requests). What you enter in the calculator is not sent or stored; seeing the result needs a one-time free sign-up.
How does the savings estimate work?
The calculator treats every request as costing the average: monthly spend ÷ monthly requests. It then applies three reductions in a fixed order: caching first, then routing to the requests that are left, then the optional retry-waste step. The example column uses the calculator's starting values, which are examples, not benchmarks.
| Step | Formula | Example (starting values) |
|---|---|---|
| 1. Share answered from cache | Cacheable % × cache hit rate % | 20% × 50% = 10% |
| 2. Spend after caching | Spend × (1 − share answered from cache) | $10,000 × 0.9 = $9,000 |
| 3. Routing factor | 1 − routable % × (1 − cheaper model cost %) | 1 − 40% × (1 − 10%) = 0.6 |
| 4. Spend after caching, then routing | Spend after caching × routing factor | $9,000 × 0.6 = $5,760 |
| 5. Spend after avoiding retry waste (optional) | Spend after step 4 × (1 − waste %) | $5,760 × (1 − 2%) = $5,645 |
| 6. Monthly and yearly savings | Spend − spend after step 5; yearly = monthly × 12 | $4,355 a month, $52,262 a year |
| 7. Cost per 1,000 requests | Spend ÷ requests × 1,000, before and after | $5.00 before, $2.82 after |
Two assumptions sit underneath. First, that repeated requests cost about the same as the average request; if your repeats are short, cheap prompts, real savings will be lower. Second, that routable requests are spread evenly between cached and uncached traffic, so routing applies to the same share of what remains after caching.
Where do the savings come from, and what are their limits?
Each lever saves money only under certain conditions, and each has a cost the calculator does not count. Treat the result as a ceiling to test, not a promise.
| Lever | Where the saving comes from | Limits and trade-offs |
|---|---|---|
| Exact-match caching | Identical requests are answered from stored responses instead of calling the model. | Only helps prompts that repeat word for word. Chat with history, personalized prompts and timestamps rarely match. Cached answers can go stale. |
| Semantic caching | Requests that mean the same thing are matched by embedding similarity and answered from cache. | False hits: two prompts can look similar but need different answers. A loose threshold returns wrong answers; a strict one hits less. Each lookup adds an embedding call and storage. |
| Routing to a cheaper model | Simpler tasks go to a smaller, lower-priced model. | Quality can drop on harder requests. You need an evaluation set to decide what is routable, and someone to re-check it when models change. |
| Fallbacks instead of retries | A failed or rate-limited call goes to another provider instead of being retried blindly. | Only saves the spend on calls that fail and are retried at full cost. A fallback model may answer differently from your primary one. |
Caching only helps when your traffic repeats. A support bot answering the same handful of questions may have a high repeat rate; an agent working on unique documents may have almost none. If your provider offers its own prompt caching for repeated prompt prefixes, that is a separate discount on input tokens and is not modeled here; see the provider pages in our LLM API pricing guides.
How do you measure your own cacheable share from logs?
Your logs give a far better number than any example. A simple way to measure it:
- Export a typical period of requests. Two to four weeks of prompts from your application logs or an observability tool, with the model used and the tokens or cost of each call.
- Normalize the prompts. Strip parts that change on every call but do not change the answer, such as request IDs, timestamps and whitespace.
- Count exact repeats. Hash each normalized prompt (with the model and settings) and count how many requests share a hash with an earlier one. That share is your exact-match cacheable %.
- Estimate near-repeats carefully. Embed a sample of prompts, group close matches and read a few dozen pairs by hand. Count only pairs where the same answer would be correct.
- Weight by cost. If repeats are cheaper or dearer than average, adjust: the share of spend that repeats matters more than the share of requests.
| Input | Where to find it | What to watch for |
|---|---|---|
| Monthly spend | Provider billing dashboard or invoice | Include every provider and API key the application uses. |
| Monthly requests | Application or gateway request logs | Count API calls, not user sessions; agents make several calls per task. |
| Cacheable % | Hashed and clustered prompts from your logs | Do not count prompts that only look alike. |
| Hit rate | A cache trial on a sample of traffic | Cache lifetime (TTL) and evictions lower the hit rate. |
| Routable % | An evaluation set scored on both models | Decide the quality bar before you look at the cost. |
| Retry waste % | Error and retry counts in your logs | Count only retries that were billed. |
For tracing and evaluating requests, see Perimattic AI Suite, and to estimate what observability itself costs, the LLM observability cost calculator.
How does Perimattic LLM Gateway help?
Perimattic LLM Gateway sits between your application and model providers. It offers:
- Exact-match and semantic caching of responses.
- Routing rules that send each request to a model by cost, latency or task.
- Automatic fallback when a provider errors or rate-limits.
- Cost tracking and attribution by team, feature and customer, with alerts.
- Budgets and rate limits per team, API key or customer.
- One OpenAI-compatible API in front of multiple providers, run in your own cloud (VPC) or managed.
Get started: Sign up and our team will get in touch, or read the LLM Gateway overview.
More on cutting LLM costs
Savings calculator questions
Does the calculator include the cost of running a gateway?
No. The figures are estimates before gateway costs: hosting or subscription fees, cache storage, and the embedding calls a semantic cache makes are not subtracted. Take your own estimate of those costs off the monthly savings to see the net figure. No vendor prices are built into the calculator.
Why is caching applied before routing?
A request answered from the cache never reaches a model, so it cannot be routed. Applying caching first and then routing only the requests that remain avoids counting the same request as saved twice. Routing alone is also shown for comparison.
Do I need to sign up to see the result?
Yes, once. You can fill in every field freely; a one-time free sign-up shows your result, and it stays unlocked in this browser after that. The numbers you enter are worked out in your browser and are not sent or stored; only the sign-up form is sent.
Are the starting values typical for most teams?
No. They are example values that give the calculator a worked starting point, not benchmarks or industry averages. Repeat rates, hit rates and how much work a cheaper model can take vary widely between applications. Measure your own from request logs and replace every value.
Want one API for every model, with costs under control?
Sign up to tell us which models and providers you use, and our team will get in touch to set up the LLM Gateway for your stack.
Sign up