A recommendation sitting unread in your dashboard doesn't save anything. Each type Preto surfaces has a slightly different implementation path — here's the workflow for each, plus how to confirm the savings actually landed once you've shipped the change.

TL;DR

1. Four recommendation types show up most often — model routing, caching (provider-native and duplicate-request), system prompt trims, and batch-eligible traffic.
2. Each has a specific, bounded implementation step, usually a day or less of work.
3. Mark it as done in the dashboard once shipped — that's what lets Preto's before/after comparison confirm the savings and feed your Estimated Savings total.

The Four Recommendation Types

Model Routing

"Route this endpoint to a cheaper model"

The recommendation identifies a specific endpoint where task complexity doesn't match the model currently handling it — usually classification, extraction, or short-form generation running on a frontier model.

  1. Review the specific endpoint and the traffic sample the recommendation is based on.
  2. Test the suggested cheaper model against your own eval set or a sample of real recent requests, not just the recommendation's example.
  3. Deploy to a small percentage of traffic first if your infrastructure supports gradual rollout; otherwise deploy and monitor closely for the first few hours.
  4. Mark the recommendation as implemented once quality holds and traffic is fully migrated.

See the full breakdown of why this pattern is so common for the reasoning behind why this is usually the highest-impact recommendation type.

Caching

"Enable prompt caching" or "X% of requests are duplicates"

Two distinct sub-types show up here: provider-native prompt caching (your system prompt isn't structured to cache well) and duplicate-request caching (a meaningful share of your traffic is repeating identical or near-identical requests).

  1. For provider-native caching: restructure the prompt so stable content sits at the top and dynamic content (user input, timestamps) sits at the bottom — the recommendation typically flags the specific prompt causing cache misses.
  2. For duplicate-request caching: implement exact-match caching first (lower risk), then semantic caching if the recommendation indicates meaningful near-duplicate volume.
  3. Set a TTL or invalidation strategy appropriate to how quickly the underlying data changes.
  4. Confirm hit rate is moving in the dashboard before marking as done — this one has an immediately visible metric to check against.

The full caching implementation guide covers the code for both approaches in detail.

Ready to work through your own recommendation list?

Start with the highest-impact one and work down.

Open Your Recommendations

Each recommendation type has implementation steps. Start with the biggest savings.

System Prompt

"This prompt has removable bloat"

The recommendation flags a specific system prompt where token count looks disproportionate to what the task requires, based on comparison against similar-purpose prompts in your traffic.

  1. Pull the flagged prompt and count tokens with a real tokenizer, not a word count.
  2. Identify stale few-shot examples and redundant formatting instructions — candidates for removal.
  3. Remove one section at a time, re-testing against your eval set after each removal.
  4. Mark as done once you've settled on a final version; the dashboard will show the reduced average token count on that endpoint going forward.

The full audit process walks through this in more depth, including the math on what bloat costs at scale.

Batch-Eligible

"This traffic could run on the Batch API"

Flagged when a meaningful share of an endpoint's traffic doesn't appear to be latency-sensitive — internal jobs, scheduled tasks, or backend processing that's currently running through the same real-time path as user-facing calls.

  1. Confirm the flagged traffic is genuinely non-realtime — a human isn't waiting on the response.
  2. Collect the eligible requests into a batch job format rather than sending them synchronously.
  3. Submit and process asynchronously — poll or use a webhook for completion rather than blocking.
  4. Mark as done once the migration is complete; the discount applies automatically at the provider level, and your dashboard will reflect the lower per-token cost on that traffic.

The full batch API guide covers exactly what qualifies and what doesn't.

Verifying the Savings Actually Landed

Once you've marked a recommendation as implemented, Preto's before/after comparison tracks the specific metric it targeted — cost on that endpoint, cache hit rate, average tokens per request — using a matched time-of-week comparison so normal traffic variance doesn't distort the read. This is also what feeds the Estimated Savings number on your main dashboard: it only grows from recommendations that are marked done and have a confirmed before/after delta, not from projected or hypothetical savings.

When to Dismiss Instead of Implement

Not every recommendation fits every situation — a compliance requirement, a quality bar the recommendation's traffic sample didn't fully capture, or a planned deprecation of the affected feature are all legitimate reasons to dismiss rather than implement. Recommendations are generated from patterns in your traffic, not from full context on every constraint your team is operating under. Dismissing one that genuinely doesn't apply is the correct call — chasing every recommendation regardless of fit optimizes the wrong thing.

New recommendations surface continuously as your traffic evolves, not just as a one-time batch at signup — working through the current list is never really "done," it's an ongoing habit that pairs well with a regular weekly review.

Frequently Asked Questions

Which type of Preto recommendation should I implement first?
Whichever has the highest projected dollar impact, unless it also carries meaningfully higher effort or risk. Model routing and prompt caching are typically both high-impact and low-effort, which is why they're usually worth doing first.
How do I verify a recommendation actually worked after implementing it?
Compare the specific targeted metric before and after, using a matched time-of-week comparison to control for normal variance. The dashboard's before/after view does this automatically once you mark the recommendation as done.
What if a recommendation doesn't apply to my situation?
Dismiss it. Recommendations are generated from traffic patterns, not full context on every constraint your team operates under — a compliance requirement or quality bar the sample didn't capture is a legitimate reason to skip.
How often does Preto generate new recommendations?
Continuously, as traffic patterns change or new waste sources emerge — not just once at signup. Expect new recommendations to surface over time as your underlying traffic shifts.

Work through your recommendation list, starting with the biggest number.

Every recommendation type has a bounded implementation path — most take a day or less to ship.

Open Your Recommendations

Questions about a specific recommendation? Reply to your welcome email.

Gaurav Dagade
Gaurav Dagade

Founder of Preto.ai. 11 years engineering leadership. Previously Engineering Manager at Bynry. Building the cost intelligence layer for AI infrastructure.

LinkedIn · Twitter