Session-store rows carried output only for compaction requests; every
other request's output was left to the per-turn assistant.message or the
shutdown rollup residual. CLI 1.0.8x and ACP hosts (JetBrains AI Chat via
copilot --acp) write no per-turn output, and an ACP session usually ends
without session.shutdown, so its whole output was lost. Rows now carry
their own output_tokens; at serve time a row that pairs with a per-turn
call carrying output (older CLIs) serves with output 0, so output counts
once in every shape.
grok-4.6 priced at $1.25/M input: LiteLLM has no bare grok-4.6 row and
the stripped azure_ai/grok-4.6 row claims the bare id ahead of
xai/grok-4.6, in the bundler and the live fetch alike. A builtin price
pins xAI's list rate ($2/$6/$0.50 per M), which is what GitHub bills.
On real data every Copilot model's cost now equals GitHub's
total_nano_aiu to the digit.
* fix(copilot): count VS Code agent-host requests and their output
VS Code now runs Copilot's agent through the Copilot CLI engine. Its
assistant.message events carry no outputTokens (CLI 1.0.8x writes the same
shape), so every request was skipped and the shutdown rollup, which assumed
per-turn events own output, dropped it: 0 calls and 0 output.
Tokenless messages now count as calls with zero tokens, and a leg with no
per-turn output takes its output from the rollup. The rollup also carries
modelMetrics totalNanoAiu, so the session's credits are exact. Tokens and
credits appear when the session ends.
* test(copilot): serve an open agent-host session
* fix(copilot): serve store-session output via the residual, count credit requests
CLI 1.0.8x sessions with session-store rows lost their output: the rows
replace the rollup and carry output only for compaction requests. The
per-leg residual now carries rollup output minus compaction-row output.
The credits note counted store rows, rollups and per-turn twins as separate
requests. It now counts requests by behavioral weight, rated when the
session carries an exact figure.
* fix(copilot): keep paired store-row credits, running nanoAiu delta
A behavioral store row (compaction or crash tail) made the credit sum drop
every paired row. Store rows now always count; a rollup counts only when
nothing behavioral carries nanoAiu.
totalNanoAiu is a running bill that does not reset at compaction, so its
per-leg delta uses its own sentinel instead of the token reset.
copilot's PENDING_REDERIVE contract moves to 56: a store row and its
message straddling midnight can move one call to the next day.
* Add regressions for redirected editor storage roots
* Honor redirected data roots for Copilot and Cursor
* Verify redirected Copilot OTel discovery on Linux
* Reproduce Cursor cache collisions across storage roots
* Bind Cursor result caches to their database and backfill redirected history
* Verify redirected discovery against the latest daily cache
* Update env guard and document editor storage overrides
* Add regressions for request-level Copilot journal usage
* Read request-level input tokens from Copilot journals
* Refresh cached Copilot journal usage and settled days
* Use local date keys in the journal cache regression
* feat(copilot): legacy JSON chat session parser and auto-mode resolution
* fixed copilot naming
* revert: remove unrelated jetbrains and env-isolation changes
* revert priority change
* fix(copilot): route legacy JSON chat sessions cleanly and fix calculateCost web search param
* fix(copilot): legacy chat session parsing and pricing
* fix(copilot): strip github.copilot-chat/ namespace from legacy model ids
found github.copilot-chat prefix in my chat sessions
* fix(copilot): revert comments
* fix(copilot): model names
* fix(copilot): model display name edge cases
* feat(copilot): only keep legacy parser source
* fix(copilot): legacy sessions skip binary tool outputs and use recorded thinking tokens
A screenshot or other binary tool result was base64-decoded as UTF-8 and
char-counted as input, so one 200 KB PNG billed 50k input tokens. Only
text/* payloads count now.
A round's recorded thinking.tokens was compared with the char estimate of
the same text via Math.max, so the estimate won whenever it was larger.
Each round now uses its recorded count when present and the estimate only
otherwise, summed across rounds.
Docs list the flat .json files and the transcript gate covering them.
* fix(models): price Copilot's dotted spellings of Claude 3.5, 3.7 and Opus 4.1
Copilot writes claude-3.5-sonnet, claude-3.7-sonnet,
claude-3.7-sonnet-thought and claude-opus-4.1, which resolved to no
pricing row and billed $0. They alias to the dashed rows now.
---------
Co-authored-by: Resham Joshi <65915470+iamtoruk@users.noreply.github.com>
Issue #1306 is about the macOS menubar's rungs. The CLI adapter gaining the
environment and gh rungs is a behaviour change it did not ask for: a shell with
GITHUB_TOKEN exported, or a gh login without Copilot, would start reporting a
fetch failure where `codeburn quota` reported "disconnected".
src/quota/copilot.ts and its suites go back to main untouched, including the
hostname character check the endpoint helpers already had. CHANGELOG and docs
drop the CLI claims.
The CLI adapter read only the editor-plugin files, so `codeburn quota` and the
Windows Capacity Dock — which has no Settings screen to fall back on — could
not see an enterprise credential that lives anywhere else. It now walks the
same rung order as the menubar for the sources a CLI can reach: the plugin
files, then COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN paired with
GH_HOST, then gh's own hosts.yml.
The gh rung reads the oauth_token out of that file rather than spawning gh, so
the token and the host come from one entry and a keyring-backed login simply
has no token there and falls through. The pasted-token rung stays macOS-only:
it lives in that app's Keychain item, which has no CLI equivalent.
Note the behaviour change this implies: a shell that exports GITHUB_TOKEN, or
a gh login with no Copilot entitlement, now reports a Copilot fetch failure
where it previously reported "disconnected". That is the same answer the
menubar has always given for those credentials.
The existing suites pin env and ghConfigPaths explicitly so a developer's own
GH_TOKEN or gh login can never change what they read.
Closes#1306
Both Copilot quota readers hardcoded
https://api.github.com/copilot_internal/user and dropped the host their
credential was read from, so an enterprise on a GitHub Enterprise Cloud
data-residency host (<tenant>.ghe.com) could only ever report
"Temporarily unavailable": its token is not a dotcom token, and dotcom is
not an endpoint it may be sent to.
A discovered credential now carries its host. hosts.json is keyed by
GitHub host and newer apps.json files key by "<host>:<app id>", so the
token and the host always come from the same entry; the rungs that carry
no host of their own (an app-name apps.json key, COPILOT_GITHUB_TOKEN /
GH_TOKEN / GITHUB_TOKEN, gh auth token, a pasted token) mean dotcom. The
endpoint follows the host: api.github.com for github.com, and
api.<tenant>.ghe.com for an Enterprise Cloud host. With several hosts
signed in github.com wins, otherwise the first .ghe.com tenant in sorted
order, which keeps the pick stable across reads; with exactly one, that
host is used.
A host neither rule can address, such as a self-hosted GitHub Enterprise
Server install, fails with a message naming that host instead of falling
back to dotcom, and network and HTTP failures name the host that was
tried. macOS Settings now says which host answered in the Copilot
connection row. Tokens are still never logged and sanitizeForUI is
unchanged.
Closes#1286
Only Copilot CLI session-store rows carry total_nano_aiu, so a typical user
has a handful of exactly rated requests and hundreds of unrated ones, and
codeburn plan showed the exact-only sum as if it were the whole bill.
Price every unrated request from its tokens at listed API rates (1 credit =
$0.01) and carry that as estimatedCredits alongside creditRatedCalls /
creditUnratedCalls. A session that already carries an exact figure is never
estimated on top of: total_nano_aiu bills that request's whole token set,
JSONL twin included. spentCredits, the bar and percentUsed stay on GitHub's
exact figures. The plan headline, and status JSON via creditsNote, now say
how much of the total is estimated.
The Copilot CLI's Keychain item is written by a Node keyring library, so
/usr/bin/security is not in its partition list. Unlike the Claude Code item,
every read would raise the macOS "allow access?" dialog, and with the probe
cache a user who clicks Deny would be asked again every five minutes.
`gh auth token` already reaches the same users, because the Copilot CLI itself
falls back to gh, so the rung was risk without benefit.
The chain is now legacy plugin files, ~/.copilot/config.json and settings.json,
the environment variables, `gh auth token`, and a pasted token. No `security`
subprocess remains in the adapter, and the probe cache collapses to the single
gh answer.
Modern Copilot clients no longer write ~/.config/github-copilot/hosts.json or
apps.json, so the menubar reported "No GitHub Copilot credentials found" even
after signing in through VS Code and the Copilot CLI.
Discovery now walks an ordered read-only chain, first hit wins, every rung
tolerating an absent, locked, or malformed source: the legacy plugin files,
the Copilot CLI keychain item (service copilot-cli, read via /usr/bin/security
so no partition-list prompt can fire), ~/.copilot/config.json and
settings.json, COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN, `gh auth token`,
and finally a token pasted into Settings. The two subprocess rungs are cached
for five minutes so eligibility checks and the 30s refresh cadence never spawn
per call, and hasCredential stays spawn-free.
VS Code's encrypted secret storage is deliberately untouched.
The no-credentials message now says usage tracking still works and names every
way to connect, and the Copilot settings pane gains a token field.
Spent credits are finite nanoAiu / 1e9. Token-priced costUSD does not
fill the Copilot bar. Thread nanoAiu through cachedCallToApiCall with
no cache-version bump.
Merge resolution
- daily-cache: main shipped v20 with #1040, so 21 is now the first FREE number
rather than one dodging an unmerged head. Both version notes kept, ours on
top. The re-derivation seed stays one below current (20), now the shipped
predecessor rather than a draft.
- upgrade-path: expected daily cache filename stays v21.
The rollup is cumulative across resume legs — already handled, now pinned
Measured on a real 3-leg session (CLI 1.0.80): every counter in a leg includes
the legs before it, and the last leg of a complete session equals its store-row
total to the nano. The parser has always emitted `cumulative - previous
cumulative` per model, so the interval arithmetic downstream already consumes
per-leg claims; it was the PR body and the provider doc that described the raw
journal as per-leg and misled the reader. Doc corrected, and (c3) pins it end
to end: a complete 3-leg session serves 900 (its last leg / store total), not
1,400 (the sum of its legs); an uncovered one serves 900 exactly once; and a
leg reporting LESS than its predecessor is taken as a fresh epoch rather than
clamped to a negative delta, so an older per-leg CLI never loses a leg.
initiator = 'compaction' replaces the timestamp heuristic where it exists
The summarization request does write its own assistant_usage_events row, and
newer stores label it. The label is now read — schema-adaptively, and the
enrichment select is a graduated chain so a store carrying the billing columns
but not `initiator` keeps its billing metadata instead of falling all the way
back to the base select. Two uses: the row is subtracted from the leg it
belongs to even though it commits before the compaction stamp, and it is kept
out of per-turn pairing since it has no assistant.message to pair with.
Optional twice over — absent on older stores, NULL on 1,504 of 2,509 rows on a
real one — so (c5) pins the labelled path at 350, the identical UNLABELLED
fixture at 400 (the documented one-request-per-compaction over-serve, which no
timestamp rule can close: the request that triggered the compaction completes
immediately before it too), and a compaction row never stealing a pairing
partner.
Attribution invariant, standing guard for the 1.8x report
(c4) runs `codeburn audit`'s own two numbers — attributed vs recomputed —
over all four combinations of the three representations a session can be
written in, at the magnitudes of a real reported day (gpt-5.6-terra, 146 rows,
input 17,792 / cache write 501,395 / cache read 12,097,364 / output 63,344 /
reasoning 24,831, billed $4.47). Every shape reconciles at 1.000, and the
covered case lands on $4.4687 = what the tokens price at = what GitHub billed,
with output 63,344 rather than the pre-fix 88,175 that re-priced reasoning on
top of itself.
upgrade-path: a day KEY is not the unit of never-lose
A parse change that re-dates a call to its true day legitimately empties one
day and fills its neighbour, token for token — observed on a real cache where
2026-08-08's single call moved to 08-07 exactly. The aging step now compares a
+/-1 day WINDOW rather than the day alone, so only a window that shrinks is a
loss, and an emptied day key is reported rather than failed.
RECONCILE_SETTLE_MS keeps 24h, with the measurement that argues against it
recorded at the constant and in the provider doc: across 91 real sessions zero
rows landed after shutdown (median -0.1s, max -0.0s). One machine, one CLI
version — the number to beat is seconds, not hours, once a second machine
agrees.
B-1 The freeze was one-directional and left the reverse hole wide open. Rows
sync; at the 90-day durable age-out the cached rows are pruned, the rollup
stops being dropped and serves again under a key that was never sent; the
session is long past settle, so it pushes an aggregate on top of the
per-request spans the receiver already holds.
Reframed around the thing that actually matters: a copilot session's
input/cache leaves this machine in one of two SHAPES — the raw rollup
(`copilot:<sid>:shutdown:`), or reconciled output (rows plus
`:shutdown-residual:`, which are disjoint by construction and together are
exactly the rollup re-expressed). The receiver must never hold both. Whichever
shape a session was first synced in, it stays in, and the other is frozen for
that session permanently — in both directions. Growth WITHIN the sent shape is
untouched, because same-shape output is additive, never substitutive: a
resumed session's new rows and residuals still push if rows were sent, a new
leg's rollup still pushes if rollups were.
This also corrects the previous commit, which classed a residual as aggregate
and so froze it for a session whose rows had gone out — the residual is
reconciled output and belongs with the rows.
B-2 A session with an unparseable timestamp was held forever while the CLI
promised it would "push once it settles". Nothing could ever settle it. Settle
now reads the newest moment a session can be SHOWN active; a session with
nothing datable at all is sent rather than held. Mixed sessions still settle on
their datable stamps.
B-3 A stamp implausibly far in the future is broken data, not evidence the
session is live, so it no longer counts toward that newest moment. One
year-ahead row can no longer hold a month-old session hostage. Ordinary clock
skew is absorbed by a one-hour grace in both directions and still reads as
live.
B-4 The residual dedup key carried the leg's POSITION. Legs sort across every
cached file for a session, so an earlier leg arriving later renumbers every
residual after it — and a renamed key is a span the receiver takes a second
time, since there is no retraction for a usage span. Keyed by the leg's own
instant instead: append-only files mean it never moves, and the equal-timestamp
coalescing above makes it unique per leg. Residuals are new in this change and
have never shipped, so no CACHE_VERSION concern.
B-5 The dry-run "already synced" count now subtracts frozen too, matching the
nothing-to-push line.
Tests: symmetric freeze from the rows side (rollup frozen, residual and new
rows still sent, per-turn untouched); all-unparseable sent, partly-unparseable
still held; a year-ahead stamp ignored while a one-minute skew still holds;
residual keys named after their leg instant with the positional names gone. The
key-prefix pin picks up the residual's new tail. (c2) pins the property rather
than one insertion scenario, because a second file's leg currently collides on
the rollup's own dedup key before it can reach the residual sweep — the
reachable repro would prove nothing about the next one, and that is stated at
the test.
Flakes: (f2) and (sc) re-bucket each run 10x isolated and 10x with four busy
loops pinning cores — 40/40 clean. Neither has clock or ordering dependence at
its margins: afterEach removes both tmpHome and the cache dir, and the two
relative timestamps in the re-bucket test are 35 days apart so they cannot
share a month. The likeliest cause of the transient failures is this branch's
own commits rewriting src/parser.ts under a concurrent vitest. (f2) gains a
self-check that its fingerprint sentinel really differs from the computed one,
which is the one way it could have passed while exercising nothing.
F1 — the upgrade-path guard was measuring the wrong layer. providerDetails
come from the menubar payload, and on a version bump adoptOlderDailyCaches
carries the superseded daily file forward as the baseline for exactly the days
this scenario creates, so a session-cache loss was mostly masked: reverting the
carry-forward showed a 10% dip where the real loss is 50%. It now sums
export.json's per-call copilot records (straight out of the parse), deletes the
baseline daily cache before the upgrade run so nothing else can answer, and
asserts EQUALITY rather than "did not shrink" — the failure mode on the other
side of the fix is a double. Proven: reverting the carry-forward now reports
80 calls -> 40, LOST history, and fails. (The double direction is asserted but
not demonstrated; a faithful re-keying simulation is collapsed by serve-time
dedup before it can be counted, which is noted at the step.)
F2 — sync was forward-only in the wrong direction. A receiver that took
`copilot:<sid>:shutdown:<model>:<n>` rollup spans from 0.9.20 keeps them
forever (append-once, no retraction for a usage span) and would now also
receive `copilot-store:` rows for the same sessions: permanent doubling on
every synced receiver, on the first push after upgrade. Sessions whose rollup
is already in the ledger are now FROZEN — their rows and residuals never go
out, and the receiver keeps the older, lossier number. A bounded under-count
in place of an unbounded over-count. Per-turn output spans are untouched
either way (the rollup never held output, reconciliation never touches them),
and sessions new to sync take the per-row path in full. `frozen` is reported
by --dry-run and the nothing-to-push line; docs/sync/README.md gains an FAQ
pointing at `codeburn sync reset --confirm` with the caveat that the receiver
must be cleared too.
F3 — docs/providers/copilot.md gains a Sharp edges section saying plainly that
a day sealed on a short store snapshot is a permanent UNDER-REPORT the
watermark advances past, not a stall, and that both realizable causes are now
closed (read ordering, compaction anchor).
F4 — the copilot dedup-key shapes are pinned by reading them back out of the
source, minted keys and discriminator prefixes alike, with the reason: the
durable union recognises history by key, so a changed prefix is a
CACHE_VERSION bump, not a parse-version bump.
F5 — the carry-forward comment now states its price: a bump no longer
re-derives a call it carried forward, so cost-shaped fixes land (cost is
recomputed from cached usage) but metadata and day-attribution fixes reach
only newly parsed calls.
F6 — the durable union filters at CALL level. A turn holding one cached call
beside a new one used to be dropped whole; nothing enforced the one-call-
per-turn assumption that made that safe.
F7 — the retain-source fence no longer skips a store it cannot fingerprint.
Present-but-unreadable defers, matching the classification path; only a
genuinely absent one stays a skip.
The CLI's session.shutdown rollup RESETS its counters at a successful
in-session compaction, so a leg containing one describes only its
post-compaction requests. Running the store-row subtraction from the previous
leg cancelled that leg's usage against the whole pre-compaction conversation
and left the residual short by exactly that much. With a complete store the
floor hid it; with a partial snapshot it was a permanent undercount, and a day
sealed on it never corrected. That is the half of the reviewer's blocker 2 the
read-ordering fence could not reach: no timing rule helps when the rows are
simply not there yet, and rollup > rows-in-interval is indistinguishable from
legitimate partial coverage.
Anchor at the compaction instead. Pre-compaction rows still SERVE; they just
stop cancelling usage the rollup never claimed. The maintainer's repro (row A
100 pre-compaction, sole rollup 200 for post-compaction request B, B's row
absent) now totals 300 instead of 200, and still totals 300 once B lands.
Event names taken from @github/copilot 1.0.80, not guessed:
session.compaction_start (nothing needed from it) and
session.compaction_complete, whose `success` is read and every other field
ignored — the payload also carries the summarization call's own usage, token
counts, model and trigger. Only success:true anchors; failed or absent falls
back to the previous leg; last compaction in a leg wins. The stamp rides on
the cached rollup call as `compactedAt` (add-only, validated optional).
Accepted, bounded over-serve, documented at the doc site: if the summarization
call writes its own store row it sits just before the compaction stamp, so it
falls outside the interval while the post-reset rollup counts it — one request
per compaction can serve twice. Not subtracted from compactionTokensUsed,
because the request that TRIGGERED the compaction also completes immediately
before it: any grace window wide enough to catch the summarization row also
catches a real request and turns an over-serve into a loss. The row writer is
native, so the bundle cannot settle it; a real store can.
(c1) pins all four shapes: the repro at 300 (and again through the cache), no
double once the row lands, a failed compaction ignored, and a leg with no
compaction unchanged. The existing 107 copilot tests stay green.
Resolved against main's newer invariants:
- daily-cache: DAILY_CACHE_VERSION/MIN_SUPPORTED -> 21. Main shipped 19
(#1015 grok); 18 was burned by this branch's earlier public head and 20 is
claimed by the unmerged #1040, and isMigratableCache would adopt either as
finalized without re-deriving. Header keeps main's v19 note and adds v21's.
- session-cache: PROVIDER_PARSE_VERSIONS keeps main's grok/dsh entries and
appends this branch's `-session-store-v2` to main's copilot value.
- parser: durable age-out takes main's orphan-only rule (#992).
retainWhilePresent is now redundant under it rather than load-bearing.
- audit-report: both imports (isBehavioralCall + sanitizeModelForDisplay).
- tests/parser (e), (l), (sc): re-pinned to the orphan-only age-out. A
still-discovered >90d copilot events.jsonl now keeps serving its per-turn
output alongside the store rows; the (sc) age-out subject is now an orphan.
- daily-cache-version-rederivation: seed 20 so the adjacent-draft-version
case is what the test pins.
- scripts/upgrade-path: expected daily cache filename -> v21.
The Copilot CLI and the GitHub Copilot desktop app both write
~/.copilot/session-store.db unconditionally; its assistant_usage_events
table holds one row per API request. Until now input/cache tokens for
these surfaces came only from the session.shutdown rollups in
events.jsonl, which are written only on clean shutdown (a crash loses
the whole leg's input/cache accounting) and lump each session leg into
one per-model total. The rollup also RESETS its counters at in-session
compaction (traced on a clean single-process 107-request session whose
sole rollup covered exactly its five post-compaction requests), so even
cleanly-closed long sessions were truncated; on a long-history machine
the store recovered ~35% of real Copilot spend lost to crashes and
compaction resets. The DB rows are per-request, crash-proof, and carry
real timestamps.
The store's input_tokens is cache-INCLUSIVE (input + cache_read +
cache_write), the same convention as the shutdown rollups — verified
against each row's token_details_json and by reconciling per-session
sums against the CLI's own footers and rollups across two machines
(1,380+ rows, 8 models, CLI 1.0.70–1.0.79, schema_version 6): every
divergence was a rollup gap. Emitted calls mirror the shutdown-call
contract: input/cache/reasoning only, output 0 — per-turn output stays
owned by the events.jsonl assistant.message calls.
Rollup-vs-store precedence is RECONCILED at serve time, per
(session, model), and only there. Both representations always parse and
cache; parseProviderSources aggregates the cached calls and, wherever
store rows exist for a (session, model), drops the rollup calls and
serves the rows plus per-leg RESIDUAL calls: each rollup leg subtracts
only the rows in its own interval — rows commit strictly before their
leg's shutdown line, so a leg at time T covers exactly the rows in
(previous leg's T, T] — and any remainder (per token component, floored
at zero) serves once at that leg's own timestamp. A store missing
requests a leg covered — adopted mid-session, rows pruned before ever
being read — therefore still serves that tail exactly once ON THAT
LEG'S DAY, a crash-tail row the rollup never saw can never cancel it,
and a complete store serves pure per-request granularity with every
residual retired to zero. The decision reads only cached contents, never discovery:
deleting or resetting the store changes nothing served, so finalized
daily history can never flip on an absence epoch; cached rows of a
deleted store remain the record until the 90-day orphan age-out (which
exempts still-discovered paths). The serve set is the one coherent
snapshot — nothing a writer does between discovery and a parse can
change what one pass sees — and read-time precedence heals persisted
duplication (stale epochs, runtimes without node:sqlite, restored
files) instead of preserving it, following the buildDurablePeriod
pattern.
Store rows and rollups carry supplementary accounting weight. A rollup
(or its residual) is aggregate accounting, never a request: zero
api-call/model-call/turn weight, tokens and cost fully retained. A
store row is one real request, but when it pairs with a served per-turn
call it is supplementary too; rows pair with same-model per-turn calls
by timestamp adjacency (monotone matching, tight 2-minute window — the
two are written at the same completion moment, and a wide window would
let a crash-only row pair against a neighbor whose own row is missing),
computed once over the FULL serve set so a date-range boundary that
separates a row from its call cannot double the request across adjacent
day queries. Only the unpaired rows — store-only requests, exactly
where crash-lost requests sit — count. Supplementary-only turns fold
into the nearest behavioral turn within 30 minutes; with no behavioral
turn to fold into they stay separate weightless turns, each on its own
day, with apiCalls 0 — and the session emission gate admits
usage-bearing zero-call sessions. The weight
propagates into the daily cache: aggregateProjectsIntoDays applies the
same rule to every calls counter and category-turn count it seals, so
v19 history and live summaries can never disagree about what was a
request.
A changed source whose read defers on the busy shape (locked, EACCES,
corrupt mid-replace — discovery still emits the source; only true
absence or a schema mismatch reads as absent) now marks session
hydration incomplete, so the daily backfill holds its watermark instead
of finalizing a day the deferred rows never reached; an unchanged
unreadable store defers nothing. The verdict travels with its result —
the 180s memo and the serve burst-reuse restore the hydration verdict
their cached data was parsed under, so a memoized partial parse cannot
inherit a later parse's complete — and a discovered source whose
FINGERPRINT cannot be read (EACCES on a present file) defers instead of
silently skipping, while a genuinely deleted file stays a silent skip. Copilot reasoning tokens are no longer
double-billed at the report layer: they are a subset of the output the
per-turn calls already price, and copilot joins claude in the
reasoning-inside-output case of the query-time cost recompute.
Store dedup keys are content-discriminated —
copilot-store:<sid>:<rowId>:<fnv1a64(created_at|tokens|model)> —
because AUTOINCREMENT prevents id reuse only within one database
lifetime: a same-path DB reset reusing row ids now mints new keys
instead of the durable union swallowing the new usage, while a
byte-identical re-insert still collapses (64-bit: 32-bit FNV
collisions between plausible token tuples are constructible). Every
call of a session serves under one project label resolved at serve
time — the session-state-derived label when the serve set knows it,
else the store rows' own — so neither rows cached before events.jsonl
existed nor an events.jsonl orphaned by a session-state prune can
split the session across two grouping keys.
CODEBURN_COPILOT_SESSION_STORE_DB is read but deliberately NOT
fingerprinted, per the #927 ruling (any copilot fingerprint change
drops cached entries whose path still exists, destroying pruned history
only the cache holds); the read is allowlisted in the #927 guard, and
serve-time reconciliation makes repointing safe without a fingerprint —
the new store's rows parse on sight and the old path's entries persist
as durable orphans. The copilot parse version appends session-store-v2
and the daily cache bumps v17 → v19: per-day attribution, call counts
and costs all change against pre-store builds. 19, not 18: an earlier
pushed head of this PR already claimed v18 under different accounting,
and the carry-forward would adopt those days as finalized without
re-deriving them.
Verified by A/B on snapshots of two real stores, a live SIGKILL crash
test (row present, no rollup, tokens recovered exactly), live resumes
whose warm-cache deltas matched new rows to the token, upgrade-healing
at 4,800-session scale, and serve-level regressions pinning every
maintainer finding from six review rounds: the rows-then-shutdown race,
stale-cache healing, age-out exemption, absence-epoch identity,
progressive row landing with residual retirement, behavioral weight
across all four pinned scenarios, the hydration fence, project
unification in both directions, the same-path reset, mixed
coverage (crash tail vs covered-leg gap), multi-leg residual day
attribution, range-invariant pairing, memo-scoped hydration verdicts,
and the fingerprint-failure fence.
JetBrains Copilot plugin ≤1.5.x (e.g. 1.5.59-243) stores all session turns
inside ONE large binary-framed outer Nitrite document, rather than the
per-turn {"__first__":{"type":"Subgraph",...}} blobs introduced in later
plugins (≥1.12.x, e.g. 1.12.1-251).
In the old format each assistant turn is a UUID-keyed Value entry whose
value field contains a JSON-string-escaped AgentRound record:
{"<uuid>":{"type":"Value","value":"{\"type\":\"AgentRound\",
\"data\":\"{...reply...}\"}"}, ...}
The extractResponseText depth-unescape loop already handles this one extra
level of escaping; the only gap was that extractJetBrainsDbTurns never fed
it the outer document — it only scanned for __first__/Subgraph blobs, which
the old plugin never writes.
Add a fallback that activates when the Subgraph scan produces zero turns but
'AgentRound' text is present in the raw file (old-format signal). It locates
the binary-framed outer document (UUID-keyed Value entry, hex matched
case-insensitively so an uppercase UUID does not fall through to $0), extracts
it with matchJsonObject, and passes it to extractResponseText. Because the outer
document holds every turn in one blob, this emits ONE session-level call per
document (all rounds' replies joined): cost/tokens are correct, only the
per-turn call-count granularity is coarser — an accepted tradeoff for legacy
data. MVStore keeps two identical collection copies; seenReplies dedupes them.
The fallback is guarded by turns.length === 0 so new-format sessions (whose
Subgraph scan succeeds) are completely unaffected and never double-counted.
Tests: old-format doc with multiple AgentRound rounds → 1 call whose token
count equals the two non-empty replies joined (the empty tool-call round is
excluded); an uppercase-UUID variant (fails without the case-insensitive
match); and a guard that new-format Subgraph turns are not double-counted.
docs/providers/copilot.md documents the old format and the one-call-per-session
limitation.
JetBrains Copilot has two turn shapes in the Nitrite .db:
- ask mode — the reply is a `Markdown` record's `text`;
- agent / plan mode (e.g. PyCharm agent sessions, `/plan …`) — the reply is the
`reply` field of an `AgentRound` record, and the `Markdown` record instead
holds the USER's prompt.
extractResponseText only read Markdown, so agent-mode turns yielded no reply
text: they were discovered (session/turn counts showed up) but priced at $0
because output tokens came out zero. On this machine that silently
under-counted a PyCharm session ($0 → $0.35) and several IntelliJ agent turns.
Determine the mode by the PRESENCE of an `AgentRound` record and read only that
record's `reply` (collecting every non-empty round in a multi-round blob).
Crucially, an agent blob whose reply is empty — a failed turn or a pure
tool-call round — does NOT fall back to the Markdown record, so a user prompt
is never mistaken for the assistant's output; such turns bill $0 as before.
Ask-mode blobs (no AgentRound) keep reading Markdown. Plan mode's sidecar
records — Thinking, PendingChanges (proposed diff, under `content`), AskQuestion,
Notification, SubTurn, and file-read `text` results — are never read as output.
Verified across all local stores: the two reply shapes never coexist in one
blob, so the split is unambiguous.
Tests: agent-mode reply extraction (ignoring the prompt Markdown), pure
tool-call rounds → $0, multi-round collection, and a failed agent turn → $0.
docs/providers/copilot.md documents both turn shapes and the ignored sidecar
records.
## What & why
The JetBrains Copilot plugin (IntelliJ, PyCharm, RubyMine, …) stores its
chat/agent sessions under `~/.config/github-copilot/<ide>/<kind>/<storeId>/` —
a location none of the existing Copilot sources (CLI JSONL, VS Code chat
sessions/transcripts, OTel SQLite) read. As a result all JetBrains Copilot
usage was silently uncounted in every CodeBurn report. This adds a reader for
that store so those sessions are discovered, priced, and attributed to the
right project.
## How it works
- **Reader.** The store's session content is a Nitrite `.db` — an H2 MVStore of
Java-serialized documents. It is scanned as `latin1` for byte-offset
stability: no Java deserializer, no new dependency, and it is not SQLite so
`node:sqlite` is not involved.
- **Reply text.** Assistant replies live in nested-escaped
`{"__first__":{"type":"Subgraph"…}}` blobs. The text is recovered by
unescaping one level at a time and, at the depth where the Markdown record's
`data` field is a well-formed one-level-escaped JSON document, reading it
structurally — so a reply containing its own quotes is never truncated or
duplicated (which would otherwise inflate the estimate).
- **Tokens/cost.** The store records no token counts, so output tokens are
estimated from the reply text (`CHARS_PER_TOKEN = 4`, re-decoded
latin1→utf8 so multibyte replies count by codepoint) and every call is marked
`costIsEstimated`. Failed generations (error status, no reply) are billed $0.
- **Sessions.** One `.db` holds many chat tabs; turns are grouped back to their
conversation GUID so the UI shows one session per tab, deduped by reply
content per conversation.
- **Project attribution**, most authoritative first:
1. the plugin-recorded `projectName` field (JetBrains Copilot 1.12+), joined
across kind dirs by store id — the billable turns live in
`chat-agent-sessions`, but the label is usually written into the sibling
`chat-sessions`/`chat-edit-sessions` store. Read length-delimited and
re-decoded latin1→utf8 so non-ASCII repo names round-trip.
2. the `.git` repo root of a referenced `file://` path.
3. a generic `copilot-jetbrains` bucket when neither signal exists.
The conversation title is a chat-thread name, not a project, so it is kept
out of the project field and surfaced as the session label instead.
Override the JetBrains github-copilot root with
`CODEBURN_COPILOT_JETBRAINS_DIR`.
## Docs
- `docs/providers/copilot.md` — full JetBrains section (store layout, latin1
scan, reply extraction, projectName precedence + cross-kind join).
- `docs/providers/README.md` — Copilot storage updated to note the Nitrite .db.
## How to verify
- `npm test -- copilot` and `npx tsc --noEmit` (fixtures reproduce the real
nested-escaped .db framing, including quote- and multibyte-bearing replies).
- End to end against a real install:
`CODEBURN_CACHE_DIR=$(mktemp -d) node dist/cli.js status --provider copilot \
--period all --format menubar-json`
— JetBrains sessions appear By-Project under their real repo names.
- Set `CODEBURN_COPILOT_JETBRAINS_DIR` to a fixture root to parse a controlled
store without touching the real config dir.
VS Code GitHub Copilot Chat users with no OTel agent-traces.db and no
~/.copilot/session-state/ saw $0.00 cost: codeburn never read VS Code's
core chat persistence, the only on-disk source carrying their token counts.
Add a fourth Copilot source that reads the VS Code chat delta-journals at
workspaceStorage/<hash>/chatSessions/*.jsonl and
globalStorage/emptyWindowChatSessions/*.jsonl. The files are a kind:0
snapshot / kind:1 path-set / kind:2 array-append journal; we replay it
(prototype-pollution-safe: __proto__/prototype/constructor segments are
rejected and containers use Object.create(null)) and read each request's
result.metadata.promptTokens / outputTokens (falling back to
completionTokens) and resolvedModel for pricing.
Dedup so users with multiple sources are not double-counted: prefer OTel
(skip chatSessions discovery when an OTel source is present), and skip a
workspace's legacy GitHub.copilot-chat/transcripts when that workspace has
chatSessions. New env overrides CODEBURN_COPILOT_GLOBAL_STORAGE_DIR and the
existing CODEBURN_COPILOT_WS_STORAGE_DIR keep discovery testable.
Tests cover the reporter's real request shape (promptTokens 32543 /
outputTokens 60 -> non-zero cost), empty sessions, emptyWindow discovery,
append-then-edit replay, requestId dedup, prototype-pollution paths, the
transcript skip, and the OTel-prefer skip.
Maintainer review fixes on top of the OTel cache-token work:
- Remove all DEBUG_OTEL console.warn scaffolding from parser.ts and
copilot.ts (gated but unlike the rest of the codebase).
- Parameterize the spans IN (...) query instead of string-interpolating
trace IDs.
- Fold the per-chat-span metadata query into the trace-span query to drop
the N+1 (one query per chat span -> one per conversation).
- Guard epochToISO against null/NaN/0 so a malformed start_time_ms row no
longer throws on new Date(NaN).toISOString().
- Remove dead code: parseSpanAttributes, OTelSpanRow, and the unreachable
`if (!db) return`; drop unused catch bindings.
- Note in parseProviderSources that the non-durable append path assumes
unique source paths.
- Document the OTel source in docs/providers/copilot.md: Node 22+
requirement, durable-cache monotonic totals, and the one-time
parse-version cache reset on upgrade.
Document the contributor onboarding path:
- CONTRIBUTING.md: setup, npm scripts, coding conventions, PR process,
the block-claude-coauthor enforcement, and the five providers without
test coverage today (claude, gemini, goose, qwen, antigravity).
- docs/architecture.md: 12-command CLI surface, parser pipeline, three
cache layers, 14 optimize detectors, and the mac / gnome / build
layouts with cited line numbers.
- docs/providers/: one file per provider (17 providers plus the shared
vscode-cline-parser helper). Each covers data path, storage format,
caching, dedup key, quirks, and a "when fixing a bug here" checklist.
Also fix two pre-existing documentation issues surfaced while writing
the new docs:
- RELEASING.md claimed GitHub Actions auto-publishes the CLI when a
v* tag is pushed. There is no such workflow; CLI publishing is
manual via npm publish. Updated the CLI section to reflect reality
and kept the menubar (mac-v* tag) automation accurate.
- .gitignore had CLAUDE.md unanchored, which on case-insensitive
filesystems also matched docs/providers/claude.md. Anchored to
/CLAUDE.md so the root-level memory file stays ignored without
affecting subdirectory docs.
All cited file paths, line numbers, function names, and test counts
were verified against current code (41 test files, 558 tests passing).
