"Self-hosting is cheaper" and "self-hosting is a trap" are both true, depending entirely on one variable most teams don't measure before deciding: how consistently busy the GPU actually stays. The fixed cost of running your own inference infrastructure is real and large — the question is whether your volume is high and steady enough to spread that cost across enough tokens to beat API pricing.
1. A single self-hosted GPU instance runs roughly $3,000-3,500/month all-in — and DevOps labor, not the GPU rental, is usually the largest piece.
2. At high, sustained, batched utilization, effective cost per token can beat most API pricing significantly.
3. At low or sporadic utilization, that same fixed cost is spread across far fewer tokens — making self-hosting more expensive than API pricing, not less.
What Self-Hosting Actually Costs
The GPU rental line item is the one most cost estimates focus on — and it's usually the smallest part of the real number.
That total is roughly fixed — it doesn't change much whether the GPU serves 10,000 requests or 10 million, up to its throughput ceiling. This is the core structural difference from API pricing, which is purely variable: you pay per token, regardless of volume, with no fixed floor.
Why Utilization Is the Variable That Actually Matters
Low / Sporadic Utilization
A GPU serving modest, bursty traffic — say, well under its batched-throughput capacity most of the time — is still paying the full ~$3,240/month fixed cost. Spread across a small token volume, the effective cost per token can be dramatically higher than API pricing, because you're paying for idle capacity most of the time.
High, Sustained Utilization
The same GPU, kept busy with concurrent batched requests, can push effective throughput far higher — published benchmarks show batched inference on modern hardware reaching well under $0.15 per million tokens, meaningfully cheaper than most API pricing. The fixed cost is the same; what changed is how many tokens it's spread across.
This is the part most "self-host vs API" comparisons skip: the same infrastructure decision can be either clearly right or clearly wrong depending entirely on a variable — sustained utilization — that has nothing to do with the sticker price of the GPU.
Before modeling a self-hosting decision, know your current API spend precisely.
You need real, sustained volume numbers to run this math honestly — not an estimate.
See Your Current API Costs FirstBefore self-hosting: know exactly what you spend today. Free cost breakdown.
Running Your Own Break-Even Math
The framework, not a single universal number:
1. Get your real monthly token volume — not an estimate, the actual number from your current API usage. Feature-level cost attribution is useful here too, since self-hosting decisions are usually about a specific high-volume workload, not your entire LLM usage at once.
2. Calculate your current blended cost per million tokens from actual API spend divided by actual token volume — this is more accurate than list pricing, since it already reflects your real mix of input/output ratio and model choice.
3. Estimate self-hosted fixed cost using the ~$3,000-3,500/month range above as a starting anchor for a single GPU instance, scaled to however many instances your volume would actually require.
4. Model utilization honestly — not your peak capacity, your realistic sustained average. A workload that's bursty (heavy during business hours, near-zero overnight) utilizes a fixed GPU far less efficiently than a workload with steady, round-the-clock demand.
5. Compare total monthly cost at your real volume, self-hosted fixed cost versus API variable cost — not cost-per-token in isolation, since the fixed-cost structure only pays off once volume is high enough to actually justify it.
What Doesn't Show Up in the Math
Two things this framework doesn't capture, worth weighing separately: data residency and compliance requirements that make self-hosting non-negotiable regardless of cost, and the engineering opportunity cost of maintaining infrastructure instead of shipping product — the 3-5x DevOps labor multiplier in the cost breakdown above is real money, but it's also real engineering time that isn't going toward your actual product.
For most teams under the volume threshold where sustained high utilization is realistic, API pricing plus the standard cost-optimization tactics — routing, caching, batching — gets most of the available savings without taking on fixed infrastructure cost and operational risk. Self-hosting earns its place specifically at the point where volume is high enough, and steady enough, that the fixed cost is genuinely being spread across enough tokens to win — which is a number worth calculating for your own workload, not assuming from a rule of thumb.
Frequently Asked Questions
How much does it cost to self-host an LLM on a single GPU?
Is self-hosting ever cheaper than API pricing per token?
What's the biggest cost most teams underestimate when self-hosting?
Should I self-host if I'm not sure my volume justifies it?
Know your real numbers before you commit to fixed infrastructure.
Preto shows your actual token volume and blended cost per million tokens — the inputs this break-even math actually needs.
See Your Current API Costs FirstFree forever up to 10K requests. No credit card required.
