The State of LLMOps in 2026: Tools and Trends and What Comes Next

In the space of nine weeks in early 2026, the LLMOps category rewrote itself. ClickHouse bought Langfuse. Mintlify bought Helicone and quietly put it in maintenance mode. Portkey open-sourced its entire gateway. If you evaluated LLM cost and observability tools even six months ago, the landscape you looked at no longer exists in the same shape.

Here's where every major tool actually stands right now, what those three moves signal about where the category is heading, and what's likely to matter next.

The Category Has Split Into Three Camps

LLMOps tooling in 2026 sorts cleanly into three positions, each with a different relationship to the request path.

Traditional APM extending into AI. Datadog and New Relic are adding LLM-specific tabs — tokens, latency, and cost alongside the infrastructure metrics teams already monitor. This is the path of least resistance for teams that already run Datadog: familiar interface, one more dashboard. But it's additive, not native — it complements a dedicated AI cost or quality platform rather than replacing one, and it doesn't evaluate output quality.

AI-native tracing and evaluation. Langfuse and LangSmith go deep on trace capture, prompt versioning, and evals — understanding what happened in an LLM call in detail most APM tools don't attempt. The tradeoff: most of this category observes traffic after the fact rather than sitting in the request path, which means no real-time routing or hard budget enforcement.

AI gateways. Helicone and Portkey sit between the app and the provider, actively shaping traffic — routing, caching, and cost tracking with minimal code change (typically one URL swap). This is the position that can enforce policy in real time, not just report on what already happened. It's also where Preto sits, with cost intelligence and budget enforcement layered on top of the gateway position.

OpenRouter occupies a narrower niche: it shows up most often in side projects, dev workflows, and as a long-tail-model fallback inside LiteLLM configurations — useful, but not typically the primary infrastructure choice for a production team with real cost pressure.

For a deeper look at what a gateway actually does on every request, see the architecture behind LLM proxies and gateway vs. proxy vs. router — the terminology in this space is inconsistent enough that it's worth defining before comparing tools by name.

What Changed in Nine Weeks

Three events in Q1 2026 reshaped who's building what, and it's worth understanding each one before picking a tool.

Read together, the pattern is a category maturing past the "every startup builds its own thin wrapper" phase and into consolidation — infrastructure companies (ClickHouse, a data platform; Mintlify, a docs platform) buying LLMOps capability rather than building it from scratch, while the open-source layer keeps getting more complete underneath them.

The 2026 Landscape at a Glance

What's Commoditized vs. What's Still Differentiated

Basic cost tracking — token counts, per-request cost, a dashboard showing total spend — is table stakes now. Every tool in the landscape table above has it. That was a genuine differentiator in 2024. It isn't anymore.

Model routing has followed the same path faster than most predicted. With Portkey's gateway now fully open source and RouteLLM-style approaches published and reproducible, "route simple tasks to a cheaper model" is close to a solved, expected capability rather than a premium feature.

What's still genuinely differentiated in mid-2026:

• Actionable, ranked recommendations with dollar estimates. Most tools show you what you spent. Very few tell you specifically what to change and how much it will save — a gap the market has been slow to close because it requires analysis, not just logging.
• Budget enforcement with a hard cutoff. Alerting is common. Actually blocking a request at the proxy layer when a workspace exceeds its monthly cap — the feature that prevents a $9,200 overnight burn rather than just emailing you about it the next morning — is rarer than the category's marketing suggests.
• A savings metric as a first-class dashboard number. "Money saved this month: $4,234" is a fundamentally different framing from "requests per second" or "total spend," and almost nothing in the landscape leads with it.

How Big Is This Market, Actually?

Two numbers frame the opportunity. Fact.MR's AI FinOps and Inference Cost Optimization Services market report forecasts a 41.5% CAGR from 2026 to 2036, reaching a USD 21.8 billion absolute opportunity by 2036 as enterprises move AI workloads from pilot to production and need cost control at the token and GPU level. For scale, Gartner forecasts USD 2.59 trillion in total global AI spending for 2026 — the cost-management layer is currently a small, fast-growing slice of an enormous and still-accelerating number.

The growth math is straightforward even without the forecasts: model prices per token keep falling roughly 50% a year, but usage keeps growing faster as teams ship more AI features and chain more calls into agentic workflows. Falling unit prices and rising bills at the same time is exactly the condition that makes a dedicated cost-management layer necessary rather than optional.

What Comes Next

How to Evaluate a Tool in This Landscape

With the category this consolidated, picking a tool is less about feature checklists and more about matching a tool's position in the request path to what you actually need enforced. Five questions cut through most of the noise:

Does it sit in the request path, or observe after the fact? This is the single most consequential architectural question. Gateway-position tools (Portkey, Helicone, Preto) can block a request before it's sent — enforce a budget, reroute to a cheaper model, apply a fallback. Observer-position tools (Langfuse's default mode, most APM integrations) see the request after it already happened, which is enough for analysis but not for prevention. If runaway spend is your actual fear, the position matters more than the feature list.

Is the roadmap still moving? Helicone's maintenance-mode status doesn't make it a bad tool today — the existing feature set is mature and the security/bug-fix commitment is real. But if your evaluation horizon is 18-24 months and you expect new capability (multi-provider routing improvements, new budget controls, new model support timing), a frozen roadmap is a real constraint worth pricing in, not just a footnote.

What's the self-hosting story, and do you actually need it? Langfuse's full MIT release changed the calculus for teams with data residency or compliance requirements — self-hosting the complete product, not a stripped-down free tier, is now genuinely on the table. For teams without those constraints, this matters less than the marketing around open source suggests; a managed SaaS with good SLAs is usually simpler to operate.

Does pricing scale with what you'd actually optimize, or with vanity metrics? Seat-based pricing on an infrastructure tool your whole engineering org touches gets expensive fast and doesn't align incentive with the thing you're buying the tool to do. Usage-metered pricing (Langfuse's units model) or free-until-a-real-usage-threshold (most gateways) tends to track better with actual value delivered.

Does it tell you what to do, or just what happened? This is the gap the market still hasn't closed. A dashboard showing $34,000 spent on GPT-5 this month is a fact. A ranked list showing which 12% of those requests could move to a cheaper model and save $8,200/month is a decision. Most of the landscape still stops at the first one.

The Wedge in a Consolidating Category

The acquisitions matter less for who bought whom and more for what they reveal: infrastructure and documentation companies now consider LLM cost and observability tooling valuable enough to acquire outright, and the open-source layer underneath the category keeps getting more complete rather than less. Both of those trends push basic observability and routing toward commodity status faster than most roadmaps assumed a year ago.

That leaves the differentiation exactly where it was pointing before the consolidation wave: not in logging what happened, but in telling a team specifically what to change, enforcing the budget that protects the change once it's made, and putting a dollar-saved number on the dashboard instead of a request-count. That's the layer Preto is built for — sitting at the gateway position with the routing and caching table stakes handled, and the ranked, dollar-denominated recommendations and hard budget enforcement as the part that isn't yet commoditized. See the seven changes that move a real bill for what that looks like applied to actual traffic.

Gaurav Dagade

Founder of Preto.ai. 11 years engineering leadership. Previously Engineering Manager at Bynry. Building the cost intelligence layer for AI infrastructure.

LinkedIn · Twitter