Commit graph

2 commits

Author SHA1 Message Date
Resham Joshi
e5dff62fa1
perf: store the session cache in day pieces (#1550)
* Store the session cache in day pieces with a per-provider index

Layout v10: a provider's files live in pieces by the UTC day of their
newest call, each with a key file, and an index row per file (piece,
fingerprint, call span, v9 month) read on every load. A ranged load reads
the index, the key files and only the members it reports on; a save
rewrites only the pieces it touched. Rows keep v9's full-load order and
month scope, so orphans, the first-paint snapshot and dedup see what they
saw on v9. The v9 month shards re-lay out once, streaming, and are removed
after v10 publishes.

* Skip cached transcripts outside a narrow query's months when unchanged

v9 could not see them, so every narrow query re-parsed each older Claude
transcript it discovered and republished the cache. The index knows their
fingerprint; an unchanged one needs nothing, a changed one still re-parses
as if uncached.

* Read piece members in batches through one buffer, and full loads concurrently

* Write piece key files without a sync; rebuild lost keys from the piece

* Changelog: day pieces

* Seed dedup with the keys of unchanged transcripts outside the query's months before a whole-file parse

* Upgrade-path check: verify the v10 envelope, indexes, pieces and key files
2026-09-24 20:45:35 +02:00
Resham Joshi
22ebfd463e
perf: load only the session-cache members a date range can use (#1542)
* Write session shards one member per line

A shard is still one JSON object, so older builds read it unchanged, but it
is written through a stream and never exists as one string. Past V8's string
limit the save used to throw and the cache silently never persisted; reads of
such a shard now go member by member.

* Hold cached files a ranged query cannot report on as stubs

A ranged load streams its in-scope shards and keeps in full only the files
with a call in the range, spawn anchors, failure markers, turn-less files and
WSL paths. Every other file becomes a stub: its fingerprint, bucket, PR flag,
dedup keys per turn and where its member line lives. A stub stands in for its
entry everywhere but the summaries it cannot contribute to: reconcile, the
seenMsgIds and parser dedup pre-seeds, the query-time key replay in serve
order, orphan surfacing, eviction and the emptiness checks. A stub whose file
changed loads its member first, so the append or re-parse runs as before, and
a wider request in the same process loads the stubs it now needs.

A bucket holding stubs is rewritten as a full load would write it, copying
the stubs' lines from the shard they came from; unchanged, the published
shard is kept. A single-line shard from an older build is rewritten on the
next save. Providers whose parse reads cached entries no report shows
(hermes, quickdesk, antigravity, devin, gemini) still load in full.

* Discover codex projects from a small index

Discovery only needs each rollout's project, but it loaded the whole codex
results file (hundreds of MB on a large corpus) on every run to get it.
flushCodexCache now also writes codex-projects.v15.json, keyed by the same
fingerprint, and discovery reads that. A missing index is built once from the
results file.

* Load the session cache once for status today and month

status asks for today, then the month. A command can now declare the union of
its ranges, so the first load already holds the month in full and the second
request finds it there; each request keeps its own month scope and summaries.

* Release the pre-lock cache view before reloading

Once the refresh lock is acquired or another process completed the refresh,
the snapshot loaded before the wait is never used again, but it stayed
referenced while the reload read a second view.

* Test the first-paint snapshot serves stubs in load order

* Parse a shard whole when the range covers all its months

A range that spans a shard's months needs nearly every member, and streaming
it one member at a time only added allocation churn (about 200MB more RSS on
report -p all over a 1GB cache). The streaming reader also reuses one read
buffer instead of allocating a fresh one per chunk.

* Parse a shard whole once the range reaches its oldest month

The previous rule also required the range to reach past the shard's newest
month, which a range ending now never does, so report -p all still streamed
every shard.

* Carry a shard line that spans reads as decoded text

Members of a megabyte and more span the 1MB reads, and joining their pieces
as Buffers allocated off-heap memory that only a GC returns: report -p all
over a 1GB cache peaked about 170MB above main in RSS. Decoding the pieces
with a StringDecoder keeps them on the heap and brings every streamed range
below main, so the whole-file parse for wide ranges goes again.

* Changelog: bounded session-cache load
2026-09-24 19:52:42 +02:00