The fork-parent memo lived in module state, so every parse worker indexed
the Codex home and read the same parent rollouts again (58 reads of 10
parents, 8 GB for 1.28 GB on a real corpus). Workers now ask the main
thread over a message port, which keeps one memo for the whole parse.
* Store the session cache in day pieces with a per-provider index
Layout v10: a provider's files live in pieces by the UTC day of their
newest call, each with a key file, and an index row per file (piece,
fingerprint, call span, v9 month) read on every load. A ranged load reads
the index, the key files and only the members it reports on; a save
rewrites only the pieces it touched. Rows keep v9's full-load order and
month scope, so orphans, the first-paint snapshot and dedup see what they
saw on v9. The v9 month shards re-lay out once, streaming, and are removed
after v10 publishes.
* Skip cached transcripts outside a narrow query's months when unchanged
v9 could not see them, so every narrow query re-parsed each older Claude
transcript it discovered and republished the cache. The index knows their
fingerprint; an unchanged one needs nothing, a changed one still re-parses
as if uncached.
* Read piece members in batches through one buffer, and full loads concurrently
* Write piece key files without a sync; rebuild lost keys from the piece
* Changelog: day pieces
* Seed dedup with the keys of unchanged transcripts outside the query's months before a whole-file parse
* Upgrade-path check: verify the v10 envelope, indexes, pieces and key files
os.homedir() reads USERPROFILE on Windows and never HOME, so the
isolation setup and every test that moved home were still resolving the
developer's real home directory. Fixtures came back empty, real sessions
leaked into counts, and the code under test wrote plans, receipts and a
sync ledger into the real ~/.config/codeburn during a run.
USERPROFILE now joins the redirected variables and a shared setHome()
helper moves HOME and USERPROFILE together; every spawn env carries both.
The remaining Windows-only failures were POSIX-shaped fixtures: chmod
tricks that only toggle a read-only attribute on NTFS, pid 1 as an
always-alive lock holder, splitting paths on '/', bare tar, mode-bit
expectations, and per-file timeouts too short for the cost of a tsx spawn.
Each gets a platform-aware equivalent so nothing is skipped on Windows and
the POSIX branches stay byte-for-byte what they were.
Full run on Windows: 267 files passed, 2 skipped, 3592 tests passed.
The Capacity Dock hover panel becomes a glance: a provider header, the
sessions running right now as pills with their context rings, a Today
strip, and a row of quota-window percentages, separated by hairlines.
Window percentages stay plain text until 70 percent so a healthy row
reads as text rather than a wall of colour. The panel stays 350pt wide;
every block height is a whole point and is summed into the panel frame,
since the hosting view does not size itself.
Running sessions are new CLI surface. `liveSessions` is add-only on the
menubar payload: sessions whose transcript was appended inside a 120s
window, carrying the branch, the model that answered last, and the
context that turn reported. Absence means "unknown", so the section
hides rather than claiming nothing is running. The block names projects
and branches, so sharing and MCP output strip it.
Today's totals needed no CLI change; the app simply was not decoding the
cache token counts the payload already carried.
Mixed-version binaries were clobbering the unsuffixed Codex, Cursor,
and Antigravity result files and each re-parsed the whole corpus.
Write *-results.v<n>.json like the daily cache. Leave the unsuffixed
file for older binaries; adopt a matching-version copy once.
Responds to ozymandiashh's CHANGES_REQUESTED review on #999.
Structural: kept the disk status-snapshot, scoped to serve's documented
CLI-spawn-fallback path, after bringing its invalidation to parity with
serve's own in-process memo (see docs/design/perf-cache-fix.md for the full
reconciliation reasoning against #972's serve.ts/watcher invalidation,
already merged into this branch's ancestry).
Correctness fixes:
1. queryKey now includes the same four pricing-config hashes parser.ts's
own cacheKey deliberately includes (proxyPaths/modelAliases/
priceOverrides/localModelSavings), so an edited pricing config
invalidates the snapshot instead of waiting for an unrelated corpus
change.
2. computeCorpusFingerprint now gives network providers (e.g.
vercel-gateway) a now-stamped synthetic fingerprint entry every call,
mirroring parseAllSessions' own treatment of the same sources, instead
of letting them contribute nothing to the hash.
3. computeCorpusFingerprint generically expands any directory-shaped
non-Claude source (e.g. mistral-vibe) via a new collectFilesRecursive
helper, instead of only Claude getting leaf-file expansion.
4. saveStatusSnapshot is now gated on isSessionHydrationComplete(), so a
degraded read-only parse can no longer be checkpointed as authoritative.
5. The settle-window defer in loadStatusSnapshot now anchors on wall-clock
time since the fingerprint FIRST mismatched (persisted via a new
mismatchFirstSeenAt field) instead of the corpus-wide newest mtime,
which fix#2 can otherwise keep perpetually "recent."
Also fixes a 6th instance of the same defect class as #1, found while
root-causing a test failure: the query key omitted the active display
currency, so switching currencies kept serving stale numbers. Updates
tests/parse-workers.test.ts, whose "warm" assertion assumed every
menubar-json call does a full parse — no longer true within the settle
window after this PR's own snapshot feature.
Three fixes from review, all measured on this box.
The workload gate was files OR bytes. The files arm is wrong: 250 pending files
holding 117 KB between them spawned 5 threads and ran ~5% SLOWER than serial,
and a file count only starts paying for itself around 400. Gate on bytes alone;
the count still takes max(files / 50, bytes / 200 MB), so a few hundred huge
rollouts keep their threads.
The flat 256 MB per-worker memory budget was contradicted by the Codex workload:
a 260 MB rollout peaks near 430 MB in its worker, linearly across the pool. It is
now derived per parse as clamp(256 MB, 2 x average pending file + 128 MB, 1 GB),
which leaves a corpus of small Claude transcripts where it was and stops
over-subscribing on rollouts. The parent's buffer of up to pool.size finished
results is part of that peak and is named in the comment.
The worker/file pairing at both install sites was positional, guarded only by
position (Claude) or a path membership check (Codex). Each worker now echoes its
path and the parent asserts it, outside the per-file try: a misalignment would
install one session's turns under another's path -- a wrong number nobody would
ever notice -- so it fails the run rather than being swallowed as a parse
failure. On the Claude side that meant hoisting the whole worker-result block
above the try, which is safe because an append never consumes a result in either
its shortcut or its straddled-fallthrough case.
A mixed Claude + Codex corpus with forked rollouts in both creation orders,
asserting an identical payload, byte-identical cache shards and a byte-identical
codex-results.json between CODEBURN_PARSE_WORKERS=0 and =3, with the codex
discard count pinned above zero so the overlap path is really exercised.
Plus: a resumable rollout never reaching a worker (the decision line reports no
full parses pending after an append), the off-thread decode matching
parseCodexFileFull exactly including the cache entry it hands back, no cache file
written by the decode itself, no leaked threads, and the files-OR-bytes gate.
The comment at the install site claimed only that an overlapping worker result
'is discarded'. State why the empty-set result is installable at all — an empty
id intersection is proof a serial parse would have dropped nothing — and why the
tempting shortcut is wrong: parsedTurnsToCachedTurns delta-encodes gitBranch
across turns, so dropping one turn changes whether a LATER turn carries a
gitBranch key. Overlap discards the whole file, never individual turns.
Tests: the end-to-end determinism check now runs both parses over the SAME
corpus, so cache shard BODIES are compared byte for byte instead of just their
keys, and a new resumed-session fixture (a transcript restating another file's
message ids, in both filename orders) makes install order decide the answer.
Verified by mutation: removing the discard guard fails it, and yielding worker
results out of order fails it.
CODEBURN_VERBOSE now reports how many worker results were re-parsed in-process
on id overlap, which is what the new test asserts on. The worker bundle's source
map is excluded from the published package (-1.8 MB).
os.freemem() reports free pages on macOS, not available memory: on an idle
128 GB machine it reads a few hundred MB, so the 2 GB gate switched the worker
pool on and off between runs on the platform the desktop app ships to. The gate
and the budget now use process.availableMemory() (cgroup/rlimit-aware in a
container), falling back to os.totalmem(): serial under 4 GB available, budget
min(0.25 * available, 2 GB). An 8 GB box earns 8 threads, a 4 GB box none.
The verbose line now carries every decision input — cores, available GB, pending
files and bytes — on both the gate and the go path, so one support log explains
itself.
Pins the gates that keep threads off low-spec machines and warm runs, that a
forced worker count bypasses them, that results come back in submission order,
that a dead pool reports failure instead of throwing (and the serial fallback
lands on the same result), that no thread outlives a parse across back-to-back
parses, and that a cold CLI parse with and without workers produces the same
payload and the same cache shards.
