4 commits

Author	SHA1	Message	Date
Resham Joshi	574a6af967	fix(pricing): take the maker's price for bare model ids (#1653) ... * fix(pricing): bare model ids take the maker's price, not a reseller's A bare id with no direct LiteLLM row went to whichever prefixed row came first in the JSON, so azure_ai priced grok-4.6 at $1.25/M instead of xAI's $2/M, and the free codestral/ row priced codestral-latest at $0. The bundler and the live fetch now let the maker's own row claim the bare id, and a $0 row yields to any priced one. Snapshot patched for the 9 keys this changes; cache schema 8 so live caches rebuild. The grok-4.6 builtin override is now identical to the snapshot and live rows, so it goes. * fix(pricing): the maker's row wins a bare id even at $0 Gemma is free on Google's own API, so gemini/'s $0 row must not lose gemma-4-26b-a4b-it and gemma-4-31b-it to a reseller's price. Priced beats $0 only among non-maker rows; codestral-latest still goes to mistral/. Snapshot now differs from main in 7 keys. Drops the stale mistral-large-latest example from the fills-only comment. * fix(pricing): pin deepseek-v3.2 output at DeepSeek's published $0.42	2026-10-06 19:56:41 +02:00
ozymandiashh	ee23e94726	Price Codex Fast (priority tier) turns at priority rates (#1628) ... * fix(codex): price Fast (priority tier) turns at priority rates Codex bills turns run under the Fast speed setting through the priority service tier, which the rollout files record per thread in thread_settings_applied events. Track that tier through the parse (and across append resumes and fork replays), mark the affected turns as fast, and derive a fast multiplier from the LiteLLM priority rate keys so those turns are priced at the priority rates instead of the standard ones. Cache versions are bumped so already-cached Codex results are re-priced. * snapshot: priority slots only, no upstream drift --------- Co-authored-by: Resham Joshi <65915470+iamtoruk@users.noreply.github.com>	2026-10-05 12:15:30 +02:00
ozymandiashh	986d72ce1e	fix(pricing): restore the DeepSeek v4 and gpt-5.6-codex repricing invariants (#1134) (#1577) ... * fix(pricing): restore the DeepSeek v4 and gpt-5.6-codex repricing invariants (#1134) The 0.9.21 snapshot caught the 2026-08-24 repricings mid-transition and relaxed two tests/models.test.ts invariants. Upstream has since synced: - DeepSeek v4: LiteLLM now carries the official peak rates for both the bare and the deepseek/-namespaced rows (pro 1.32/3.96 per million, flash 0.30/1.20), so the stale hand-pinned MANUAL_ENTRIES (from before PR #27056 merged) are removed and prefixed-equals-bare is asserted again. - gpt-5.6-codex / -max: still absent from LiteLLM, so their MANUAL_ENTRIES remain but now mirror the repriced gpt-5.6 base row verbatim (4/20 per million, tier block included). codex-equals-base is asserted again, and the long-context tier test now expects the codex SKUs to carry the base tier. The refresh also exposed a pass-2 ordering defect in scripts/bundle-litellm.mjs: a reseller row like openrouter/deepseek/deepseek-v4-pro could claim the empty deepseek/deepseek-v4-pro slot before the official row was reached (the winner was decided purely by upstream JSON key order), pricing the namespaced id at the resale rate. A prefixed row may now claim only a stripped slot that is not itself an upstream entry name; multi-segment ids that exist only as reseller rows keep resolving as before. Bundled snapshot regenerated: 24 rows moved by upstream drift (claude-sonnet-5-5 family added, deepseek-v4.1-flash repriced, zai/glm-4.6 cache-write filled), plus the deepseek/codex corrections. Token-priced calls re-price on read; no re-parse is needed. * fix(pricing): repin cline-cli parse version and cover #1134 shadow bug - session-cache.ts: bump the cline-cli parse version so cached estimated costs (e.g. offline DeepSeek v4.1 flash) re-price on read, since cline-cli is in REPORTED_COST_PROVIDERS. - CHANGELOG.md: note the bundler fix also reprices zai/glm-4.6 from the vercel_ai_gateway resale row to the official row, and moves the two mistral embed models to the fallback table. - bundle-litellm.test.ts: add a fixture covering a reseller-prefixed row that sorts before the official row it would otherwise shadow. --------- Co-authored-by: Resham Joshi <65915470+iamtoruk@users.noreply.github.com>	2026-09-29 15:05:37 +02:00
ozymandiashh	700e4d1a27	fix(pricing): carry long-context tier rates through bundling, applied where billing evidence exists (#1478) ... * fix(pricing): carry long-context tier rates through bundling and price them where billing evidence exists LiteLLM publishes *_above_<n>k_tokens rates for 418 models - OpenAI's gpt-5.4/5.5/5.6 family above 272k, Anthropic above 200k, and 128k/256k/ 512k tiers elsewhere - but the bundler flattened every entry to the fixed five-slot tuple and calculateCost had no threshold dimension, so a request past its threshold priced at the base rate forever (#1076). The tuple gains an optional sixth slot {threshold, input, output, cacheWrite, cacheRead}, the threshold parsed from the key suffix because LiteLLM ships no numeric field (272k means 272,000; #1075 verified assuming 128k fabricates +64% spend). Old bundles parse unchanged; CACHE_SCHEMA_VERSION moves to 5 (4 is taken on main by the implicit cache-write billing) so a stale on-disk live-pricing cache cannot reintroduce the old behavior; the live-fetch parser picks the same tier up; service-tier _priority/_flex variants and the 1-hour cache-write combination are not context thresholds and stay out. A tier applies when the request's prompt tokens (input + cached input, the existing grok-4.6 definition) reach the threshold, each published rate replacing its base and omitted slots keeping the base. Application is gated on billing evidence via TIERED_PRICING_PROVIDERS: codex today, where the maintainer's corpus measurement shows the surcharge passing through; a real Copilot session on gpt-5.6-terra with ~6M-token prompts billed at the BASE rate, so Copilot keeps base pricing until such evidence exists - applying it there would fabricate spend, the exact class #1075 warned about. The audit report recomputes through the same tier swap so recomputedTotalUSD keeps tracking attributedCostUSD. * fix(pricing): read tier rates only from the chosen threshold; e2e codex tier test Repair pass from review: both tierOf implementations now collect rates per threshold and read only from the largest one, so a hypothetical two-tier entry can never mix the smaller tier's rates under the bigger threshold; the bundler validates tier values exactly like the live parser (finite, non-negative). A codex end-to-end test drives a real rollout past gpt-5.5's 272k prompt threshold and asserts the tiered arithmetic explicitly (a typo'd provider string in codex.ts would leave the call at base rates and fail it), alongside the no-provider base-rate control that keeps the real Copilot billing pin intact. Comment fixes: the priceOverride wording, the grok pin's actual resolution branch, and the threading note on TIERED_PRICING_PROVIDERS for whoever adds the next provider. * fix(pricing): recompute the audit tier per call and re-derive warm daily caches Both review blockers: - audit-report no longer feeds a bucket's SUMMED tokens into the per-call 272k threshold (a \,979 phantom gap on the reviewer's corpus, any tiered codex model clearing the threshold on the sum alone). The recompute accumulates per call, through the same tier swap calculateCost applies, with each call's own prompt total; a new fixture mixes one 300k call with 200 small ones whose sum blows past the threshold and pins that only the big call tiers. Web search stays on the base rates - it never participates in a tier. - DAILY_CACHE_VERSION 33 -> 34 with MIN_SUPPORTED_VERSION raised to match, the cfdaa685 convention: a 33 file written under tierless pricing is below the floor, is not adopted as the finalized base, and its days re-derive off the warm session cache (\ of codex spend on the reviewer's corpus lives only in the daily cache). The migration tests pin the new floor contract: no version-diff repairs merge into a below-floor file, and the v30 adoption test tracks CURRENT. Nits from review: the kimi precision tightening is reverted to (8, 5), and the tieredCostsFor comment is rewrapped. Measured on this machine's corpus, tier-only (branch vs the refresh branch, same data): +\.43 lifetime codex - this corpus runs far more sessions past 272k than the reviewer's 8-of-51,801 (+\.012964); both numbers are real, the corpora differ. * fix(pricing): preserve base alias selection when bundling context tiers * fix(pricing): address the 28 Sep review - floor stays at 28, tier bills the implicit cache-write rate Restore main's two carry-forward tests: with MIN_SUPPORTED_VERSION kept at 28 (per the review's correction, warm caches re-derive identically under 41/28, so raising the floor is unnecessary) a v33 file sits in the adopt-with-repairs window again and main's expectations hold; the tier's warm-cache coverage stays in daily-cache-version-rederivation. The tier-omits-slot test bills an implicit cache-write at the input rate of the costs object in effect (#1544), so the tiered expectation is 1_000 * cacheWriteCostPerToken('gpt-5.5', tieredCosts). Note on the Codex estimate path: an estimated prompt can cross a long-context threshold and tier itself; measured corpora hold no estimated calls that cross. * fix(pricing): pin claude-opus-4 primary and keep carried fallback rows hygienic in the refreshed snapshot The requested regeneration surfaced two upstream moves beside the tier work. LiteLLM dropped `claude-opus-4`; a refresh carries dropped ids into the fallback tier, but the Cursor-style alias `claude-4-opus` resolves against primary rows, so the bare id fell to the snowflake gateway row and under-priced by 3x. Pinned at Anthropic's list price in MANUAL_ENTRIES so it stays primary. LiteLLM also repriced grok-4.6's base input 2e-6 -> 1.25e-6 (tier rates unchanged); the below-threshold literal follows. Both changes reproduce identically with main's unchanged bundler on the same live responses. Today's drops also exposed a gap in the #1483 carry-forward: primary files legitimately hold @pin, date-suffixed and free-on-both rows, and carrying those verbatim imports exactly what tests/pricing-fallback-data.test.ts keeps out of the fallback file (main's own next refresh fails those tests today). The carry now skips @pin/date-suffixed keys (no query can arrive in those forms) and free-on-both rows (an unpriced model falls to expected-free handling); vendor-prefixed keys stay carriable since queries match them verbatim. --------- Co-authored-by: Resham Joshi <65915470+iamtoruk@users.noreply.github.com>	2026-09-28 19:10:39 +02:00