The provider invoice is one number. Your finance team wants it split by product line, your engineering manager wants it split by feature, and somebody in the middle wants to know why it tripled in March. If you did not attribute cost at call time, none of those questions have honest answers.
Why reconstruction after the fact fails
The usual attempt is to take application logs, join them against provider usage exports, and allocate. It fails for boring, structural reasons.
- Timing. Provider usage is aggregated on their clock in their timezone; your logs are on yours. Requests near a boundary land in different buckets and the totals never quite match.
- Retries. A retried request bills as two calls upstream but appears as one operation in your application log — if the retry happened inside an SDK, it may not appear at all.
- Streaming. A stream that the client abandoned still consumed tokens. Your log likely records a cancelled request; the invoice records the tokens generated before the cancel.
- Shared keys. Once two services use one key, the provider cannot separate them and neither can you.
Each of these is a few percent. Together they are enough that nobody trusts the resulting spreadsheet, which means it stops being maintained, which is how organisations end up with an unexplained six-figure line item.
Attribute at call time or not at all
The workable model is to decide the dimensions before the request leaves your infrastructure, and to record them on the same record that carries the token counts:
- Key — the credential used. This should be per service or per team, never shared.
- Tenant or customer — if you resell, this is the dimension your own margins depend on.
- Feature — the product surface that triggered the call, passed as a header or tag. "Summariser", "support-autoreply", "eval-run".
- Environment — production, staging, and the eval harness someone left running over the weekend.
Three of those four cost nothing to add and are impossible to add later.
Count tokens from the response, not your estimate
Client-side estimation with a tokeniser is useful for guardrails and useless for billing: it does not know about system prompts injected downstream, cached prefixes, tool schemas, or provider-side reformatting. The number that matters is the usage block the provider returns with the response.
For streaming, that block arrives at the end, and it is the single most commonly dropped field in a streaming implementation. If your stream handler does not persist final usage, your cost data has a hole exactly where your most expensive requests live. Distinguish input, output, and cached-input tokens: they are priced differently, and in cache-heavy workloads the difference between them is most of the bill.
Attach price at write time
Store the rate you applied alongside the token counts, rather than computing cost later by joining against a current price table. Prices change; when they do, a "current price" join silently rewrites history and your March number stops matching what March actually cost. Recording rate-at-time-of-call also makes your internal numbers reconcilable against the provider invoice line by line, which is what turns an argument into an audit.
Budgets are the part people actually want
Attribution answers "where did it go". The follow-up is always "stop it from going there". Once cost is attached per key, enforcement becomes possible: a monthly ceiling per key, an alert at a percentage of it, and a defined behaviour at the limit — throttle, downgrade to a cheaper model, or reject.
Decide that behaviour deliberately. Silently downgrading a model changes output quality without telling anyone, which is a support ticket wearing a cost control's clothing. Rejecting is honest but breaks the feature. Throttling is usually the least bad default for interactive traffic, and hard rejection is right for batch jobs.
Where this lands architecturally
All of the above is per-request bookkeeping on the hot path, which means it belongs in the component every request already passes through. Doing it in each service means implementing it four times and getting three different answers. That is the practical argument for a gateway — see how we handle usage and quotas, or the wider write-up on controlling LLM cost.