Inference.

Memory, latency and the cost of useful work.

Fast tokens. Slow answers.

Why a fast decoder can still leave you waiting, and the four measurements that make a comparison useful.

↗

Do the inference maths.

Idle hours, output tokens and rejected answers belong in the same budget. Start with your own assumptions.

↗