Nobody sits down and writes a bloated system prompt on purpose. It starts at 400 tokens — clean, purposeful, every line earning its place. Six months and forty small edits later, it's 2,000 tokens, and every single one of those tokens is being paid for on every single request, forever, whether or not it's still doing anything.
The fix isn't complicated. It's a manual audit, done properly, against your own evaluation criteria — not an automated tool, though those exist too. Here's the process.
1. A typical production system prompt has 25-50% removable content that stopped earning its tokens months ago.
2. The audit is four steps: count tokens, flag stale few-shot examples, flag redundant formatting rules, remove one section at a time and re-test.
3. At scale, 1,000 tokens of bloat on a single high-volume endpoint is worth roughly $1,875/month — a real number, not a rounding error.
Why This Happens to Every System Prompt Eventually
The pattern is almost universal because every addition to a system prompt has the same asymmetry: adding a line feels safe (it fixed a real problem once), and removing a line feels risky (what if it breaks something?). So prompts only grow. A few-shot example added after one bad output in March is still there in September, even though the underlying issue was fixed a different way in April. A formatting instruction added when the model ignored a JSON schema is still there after you migrated to structured outputs, which enforces the schema regardless of what the prompt says.
None of this is anyone's fault — it's just what happens when the cost of an addition is invisible (a few extra tokens, who's counting) and the cost of a removal is visible (a real, if small, risk of regression). The audit exists to make the invisible cost visible.
The Four-Step Audit
Count tokens with a real tokenizer
Pull your top 3-5 system prompts by request volume and run them through tiktoken (or your provider's equivalent) — not a word count, an actual token count. This becomes your baseline and, later, a regression-test assertion: pin the expected token count for each known prompt template and flag drift in CI.
Flag few-shot examples that haven't been validated recently
Any example that hasn't been checked against your eval set in 6+ months is a candidate for removal. If it doesn't measurably move quality when tested against current model versions, it's not earning its tokens — it's dead weight the model has to read on every call.
Flag redundant formatting and boilerplate instructions
Many "respond in JSON" or "be concise" instructions are now redundant since structured output modes enforce schema directly and length constraints can be set via API parameters. Politeness and persona boilerplate that never shows up in user-visible output isn't earning its keep either — if it isn't observable downstream, it isn't doing anything.
Remove one section at a time, re-test, keep what moves quality
Remove flagged sections individually, not all at once — batching removals makes it impossible to attribute a regression to a specific change if one shows up. Run your eval set (or a sample of real production queries) after each removal. If output quality doesn't measurably change, the section is gone for good.
Want to see your own prompt bloat?
Preto shows average input tokens per feature, so you can spot which prompts grew the most without pulling logs manually.
See Your Prompt Token Breakdown — FreePreto shows average input tokens per feature. Find the bloated prompts.
What the Bloat Actually Costs
The abstract "your system prompt is too long" argument gets more convincing with real numbers attached. Here's the math on a single bloated prompt at production scale:
× 50,000 requests/day
= 50,000,000 extra input tokens/day
× 30 days
= 1,500,000,000 extra input tokens/month
× $1.25 per 1M input tokens (GPT-5)
= $1,875/month — from one prompt, on one endpoint
That's not a hypothetical high-volume enterprise scenario — 50,000 requests a day is roughly 35 requests a minute, well within reach for a single popular feature on a mid-size product. Most teams run more than one high-volume endpoint, and most of those endpoints have some degree of prompt bloat once you actually measure it. The $1,875 figure above is a floor for a single feature, not a ceiling for the whole audit.
The Research Ceiling, for Context
If a manual audit isn't aggressive enough, Microsoft Research's LLMLingua demonstrates up to 20x prompt compression with roughly 1.5% performance loss on GSM8K, using algorithmic token removal rather than manual review. Most teams don't need to reach for automated compression to capture most of the value — a disciplined manual audit against a real eval set typically recovers 25-50% of a bloated prompt's tokens, which is most of the available savings without adding a new dependency to the pipeline.
Whichever approach you use, the pattern from the five recurring cost patterns applies here too: the audit isn't a one-time fix. Pin token counts as CI assertions, and re-run the audit whenever a prompt gets touched — otherwise the same six-month drift that created the bloat the first time will create it again.
Frequently Asked Questions
How much can a system prompt audit actually save?
Why do system prompts grow over time?
How do I know what's safe to remove from a system prompt?
What's the difference between a manual audit and automated prompt compression?
Find your bloated prompts before your next bill does.
Preto shows average input tokens per feature so you know exactly which prompts to audit first — and tracks the count going forward so drift doesn't come back unnoticed.
See Your Prompt Token Breakdown — FreeFree forever up to 10K requests. No credit card required.
