# Do the inference maths.

Published 2026-11-04 by GLM Handbook Editorial. Canonical: https://glmhandbook.dev/articles/api-vs-local-inference-cost/

An API invoice makes cost visible one request at a time. A local server hides some of it in hardware you already bought, electricity you rarely allocate and time you do not bill yourself. A fair comparison starts by making those assumptions explicit.

## Count input and output separately

Many APIs price input and output tokens differently. If a workload sends 10 million input tokens and receives 2 million output tokens, its bill is not “12 million tokens at the input price.” Multiply each category by its rate, then add the categories.

The calculator below uses editable, illustrative rates. They are not a quotation from a provider. It excludes cache discounts, fallback charges, taxes and additional services so you can see the core arithmetic. Replace every field with your own assumptions before using the result to budget.

<!-- calculator -->

## Calculate the comparison honestly

For an API, use **input millions × input rate + output millions × output rate**. For a rented GPU, include billed hours and any storage or bandwidth charges. For owned hardware, choose an amortization period and include the portion of the machine dedicated to this workload.

A machine that would be running anyway may have a low incremental cost for your experiment. A machine purchased solely for inference does not. Keep both views if they inform different decisions: incremental cost for a short experiment, fully allocated cost for running a business.

## Utilization does the heavy lifting

An hourly server billed all month costs money while idle. A heavily used system can spread that fixed cost across many tasks, but utilization does not rise by wishing it would. Estimate demand from a real usage pattern and leave room for peaks.

Batch work can sometimes use scheduled capacity and shut down after completion. Interactive work may need spare capacity so requests do not wait. Those are different operating models even when they serve the same checkpoint.

The calculator’s break-even multiple simply scales the entered token mix until its API cost equals the entered fixed monthly cost. It assumes rates stay constant and the fixed server can handle the workload. It is a first comparison, not a capacity forecast.

## Add the work around the model

Measure maintenance, deployment recovery and evaluation time. Do not invent a precise hourly cost if you do not have one: list the work and calculate a range. A local setup can be valuable because it provides control or offline access even when it is not the cheapest route.

Include the rejected answers too. A cheaper model that needs three attempts per accepted result can cost more than a more expensive model that succeeds once. For an agent, tool calls, long histories and repeated recovery steps can dominate the bill.

## Check feasibility before the price

The cheapest setup is irrelevant if the model does not fit, cannot serve the required concurrency or misses the task’s quality bar. Use the [memory guide](/articles/llm-quantization-memory-guide/) to estimate the hardware requirement and the [latency guide](/articles/llm-latency-tokens-per-second/) to measure the user experience.

Keep the final decision as a short record: workload, quality threshold, token mix, utilization, monthly cost range and the date you checked the rates. Revisit it when traffic or the serving stack changes. That makes a budget a useful engineering tool rather than a number you defend long after its assumptions expire.