Margin Protection Strategies for AI Products With Variable Inference Costs

Classic SaaS economics rest on one assumption: that the cost of serving an additional customer approaches zero once the software is built. A company writes the code once, and the thousandth customer costs almost nothing to serve beyond the marginal infrastructure of hosting and support. That assumption is why software became its own asset class, often with a substantial majority of revenue falling to gross profit at scale, because the expensive work, engineering, happened upfront and the serving cost barely moved as revenue grew.

AI products break that assumption at the root. Every query a user sends spends real inference, because it's computed against a model that charges by the token, so that cost recurs on every single request instead of fading into a fixed cost base. There is no point at which the marginal customer becomes free to serve, because the compute does not amortize away with scale, and it repeats on every request.

The numbers bear this out. AI-native products averaged roughly 52% gross margins in 2026, up from a much lower baseline in 2024, because operators got better at managing inference spend. That improvement reduces inference waste, but SaaS-style margins will not eventually arrive.

Three cost lines drive that compression, and all three move with usage rather than sitting flat on the books: per-token inference charged on every request, evaluation engineering needed to keep model outputs reliable, and observability required to monitor what the model is doing in production. None of these behave like the fixed engineering cost of traditional software. They scale with how much the product gets used. Gross margin is now a function of customer behavior in a way it never was for a database-backed SaaS app.

Agentic architectures make this nonlinear. A single chat turn consumes a bounded, predictable number of tokens. A multi-step agent session, chaining reasoning, tool calls, and retries, can consume orders of magnitude more tokens for what the customer experiences as one task, and no additional revenue attaches to that extra compute under a flat price. Falling per-token prices do not rescue the model either: as inference gets cheaper, companies push AI into more workflows and more steps per workflow, and total token consumption grows faster than the unit price falls. The problem was never only about what a token costs. It is about how much consumption shape can expand underneath a fixed price, and that shape is set by the heaviest users long before it appears in a blended average.

Power-user asymmetry and the hidden margin problem

The margin damage from inference cost does not accumulate evenly across a customer base. It concentrates in the heaviest cohort of users, and a blended average across the whole customer base can look perfectly healthy while that cohort is quietly running at a loss. A flat monthly price can earn strong margin on a median user running a moderate task load, but if a power user runs five times as many tasks through an agentic loop, that same price can collapse to near zero. The price was never wrong on average. The consumption distribution was simply wider than the pricing model accounted for, and because the heaviest users are usually the most engaged ones, the loss compounds the longer the relationship lasts.

GitHub's experience with Copilot is the clearest documented instance of this dynamic breaking a live pricing model. GitHub moved Copilot away from its request-based structure, so it now bills usage through AI Credits tied directly to token consumption. GitHub's own documentation states that the cost of an interaction now depends on the model used and the number of tokens consumed, replacing a system that counted premium requests without regard to how much compute each request actually burned. GitHub stated that flat-rate pricing could not hold up once agentic workflows began consuming far more compute per task than the original pricing model assumed.

Atlassian takes the same underlying problem and solves it through a hybrid structure with Rovo that combines a flat allowance with overage charges. As of June 9, 2026, standard plans include a set number of Rovo AI credits per user per month, and if usage goes beyond that allowance, per-credit overage charges will start on December 3, 2026. The credit allotment protects predictability for typical users, and the overage rate is the mechanism that stops the power-user cohort from quietly eroding blended margin for everyone else.

The danger in both cases is the same one: a product that bundles AI into a flat per-seat subscription can be operating at a structural loss for its heaviest users without anyone noticing, because cost tracking happens at the aggregate account level. Aggregate gross margin can look fine for several quarters while a specific cohort drags it down, masked by customers who use the product lightly enough to subsidize the ones who don't. You can't just model per-user inference cost at median and heavy-usage levels as something a finance team reports on after the fact. It has to happen before a pricing structure is chosen at all, because the gap between what a median user costs and what a heavy user costs is what determines whether any given price is safe to offer.

Modeling per-user inference cost by cohort before committing to a pricing structure

Producing a per-user cost number at both the median (P50) and heavy-user (P90) consumption levels has to come before a pricing decision, not after one. The P50 figure tells you whether the headline price covers cost for a typical user. The P90 figure answers something more consequential: whether the heaviest users in the base can erase gross margin inside a single quarter if left on the same plan as everyone else. A pricing model built only around the P50 number will look sound in a board deck and fail in production the moment usage concentrates in a subset of accounts.

Three inputs build that cost number: tokens consumed per action, the price per million tokens charged by the model provider, and the distribution of actions per user per month, measured separately at P50 and P90. Output length is a variable that gets missed constantly in these models, because longer outputs cost more per action than shorter ones, and agentic tasks that retry or re-prompt themselves stack additional output tokens in ways a simple per-query average will not capture. If a cost model runs on average tokens per query instead of the realistic distribution of output lengths across retries, it will understate true cost for exactly the workflows driving the most usage.

Cacheability belongs in the model with the same weight as raw token volume. If repeated inputs hit a prompt cache, they earn a substantial discount on cache reads, and Anthropic, OpenAI, and Google have each adopted this pricing structure independently. A cohort running mostly novel queries has a fundamentally different cost profile than a cohort running repeated system prompts or retrieval context against a cache, and that mix has to be estimated per cohort rather than assumed to be uniform across the customer base.

Model routing is a cost assumption, not a fixed constant, and it needs to be stated explicitly inside the cost model rather than left as an infrastructure detail decided later by engineering. The price spread between a mid-tier model and a flagship model from the same provider can run five times or more, and the choice between routing a task to a cheaper model or a frontier model changes the entire cost base underneath a pricing decision, so whatever routing policy a team intends to operate in production has to be built into the cost projection from the start.

The clearest tool for tying this cost modeling to a pricing gate is the Inference Efficiency Ratio: IER equals AI product revenue divided by inference cost. For AI-infused SaaS products, where AI is a feature layered on top of a broader platform, the benchmark target is an IER of 10:1 or higher, which keeps inference at roughly 10% of AI product revenue. For AI-native products, where inference is the product itself rather than an add-on feature, a 5:1 ratio is the structurally appropriate target, reflecting a business where the model call is the core deliverable. IER should be calculated monthly, tracked on a trailing three-month basis, and forecast forward against expected usage growth, because it functions as a leading signal: it will show a pricing change or a new feature eroding margin well before that erosion appears in reported gross margin.

Posted token prices from a model provider cannot be the whole model either. A cost projection that doesn't account for usage growth trajectory, output length distribution, cacheability mix, routing policy, and retry rates will look optimistic on the day it's built and fail once real usage scales, because token price deflation without volume growth accounted for produces a forecast that understates true exposure.

Investors reading a pricing page or a data room treat the absence of this number as information in itself. No per-user inference figure in the data room signals that the company hasn't measured it. A pricing page with no usage cap signals that P90 users are likely crushing margin somewhere inside the business, whether or not the company has noticed yet. The cost model built at the cohort level is what makes the next decision possible: choosing an actual pricing structure that is defensible.

Choosing a pricing structure that moves variable cost exposure toward the customer

Every pricing structure available to an AI product sits somewhere on a spectrum defined by how much variable-cost exposure the vendor retains versus passes through to the customer. That is the actual axis the decision turns on, not which structure customers say they prefer in a sales call.

A pure flat subscription keeps all of that exposure on the vendor's books. When a customer's usage spikes, whether from a new agentic workflow or simple growth in team size, the product's margin falls with no revenue offset to compensate. This is the structure underlying the power-user asymmetry described earlier, and it's inherent to the model itself rather than a sign that a particular company priced it wrong.

Pure usage-based pricing moves exposure entirely onto the customer, so it keeps margin safe, but it creates a different problem in sales. Enterprise finance teams have a hard time budgeting for software spend that varies month to month, and that unpredictability slows sales cycles and raises churn risk among customers who get hit with an invoice larger than they expected. A structure that protects margin perfectly but makes procurement harder just trades one problem for another.

Most AI companies have landed on a hybrid between the two: a flat base tier paired with usage caps and overage charges once consumption crosses a threshold. The base fee buys access and gives the customer a predictable number to budget against. Once a customer's consumption passes the point where the flat fee stops covering cost, the overage or credit mechanism captures that revenue. The cap itself is a direct output of the cohort cost model built in the previous step: the safe cap is roughly the consumption level where the flat fee still earns acceptable margin, which for most products is meaningfully above what the median user actually consumes.

Credit-based systems are a specific variant of this hybrid, useful when a product's value metric is still settling and a straight per-token charge would feel arbitrary to a customer. Miro allocates AI credits according to plan tier. Atlassian bundles 25 Rovo credits per user per month into its standard plan, with per-conversation overage charges once a user exceeds that allowance. The design risk with any credit system is that a credit becomes an abstraction customers can't translate into something concrete. The practical test is whether a customer can explain what a single credit buys without opening a help document; a credit system that fails that test generates support tickets.

Salesforce's Agentforce shows how an enterprise vendor can run more than one pricing track at once rather than forcing every buyer through the same structure. Agentforce launched with conversation-based pricing, then added Flex Credits priced per AI action in May 2025, and added traditional per-user licensing later that same year. That approach meets different buyers where their procurement processes already sit, but it needs billing infrastructure that can meter and invoice across all three tracks without forking into separate engineering codepaths for each one.

Clay introduced a structure in March 2026 that offers the clearest published separation between access pricing and inference cost recovery. Clay splits platform orchestration, billed as Actions, from marketplace data acquisition, billed as Data Credits, into two distinct billing buckets rather than folding inference cost into a single tier price. That separation makes the cost pass-through explicit to the customer rather than burying it inside a number the customer can't decompose.

One more decision belongs in this layer: what to do when a model provider drops its posted token price. The instinct to pass that savings straight through to customers gives away a margin improvement the company earned by nothing more than waiting. Holding prices steady for at least a quarter after a price drop banks that efficiency gain, and the margin it preserves can fund the routing and caching improvements that compound savings further on the cost side of the ledger.

Technical levers that reduce inference cost without degrading customer-facing quality

Pricing structure governs how variable cost gets shared with the customer. The cost itself still has to be managed on the engineering side, and two levers compress it more than any others: model routing and prompt caching. Both need to be planned as deliberate product decisions, not left to whatever a model API defaults to.

Model routing works because the price spread within a single provider's model lineup is wide: a mid-tier model can cost five times less or more than that provider's flagship. Routing routine, low-difficulty tasks to a cheaper model and reserving the flagship model for tasks that genuinely require its capability compresses an inference bill more than any single pricing change can. Mid-tier models now deliver most of a flagship model's practical capability on the kind of routine tasks that make up the bulk of real usage, at a fraction of the cost, so defaulting every request to the flagship model is a margin decision with a measurable, avoidable cost.

Prompt caching is the second lever, and it works because Anthropic, OpenAI, and Google have each converged independently on offering a steep discount for cached input reads on repeated prompts. Any product with a common system prompt, a shared retrieval context, or repeated document inputs across requests has a cacheability rate that belongs in its per-user cost model as its own line item, not folded into an average token cost. The mechanics differ by provider and have to be modeled accordingly: Anthropic charges a premium to write to the cache in the first place, while Google bills cache storage by the hour, so a cost model built on a generic caching assumption will misstate cost for whichever provider a product actually runs on.

Routing and caching get most of the attention, but output length control is just as simple to apply. Longer model outputs cost more per action, and a large share of AI features generate longer outputs than the task needs. If you enforce explicit output length constraints inside prompts, you cut token spend directly, and most use cases lose no task quality for it. Retry logic in agentic systems deserves the same scrutiny: agents that retry on failure or reason across long loops stack output tokens with no revenue attached to the extra compute, and rate-limiting retries, inserting intermediate validation steps, and designing agents to fail fast on low-confidence paths are engineering choices with direct, measurable margin impact.

None of these levers substitute for the cost model and pricing structure built in the earlier stages. Routing and caching reduce the cost side of the equation, but a company still needs to know, cohort by cohort, what a user costs before deciding how much of that cost a given price is built to absorb. The two layers work together: pricing decides who bears the exposure, and engineering decides how large that exposure gets.