mirror of
https://github.com/NeuralNomadsAI/CodeNomad.git
synced 2026-11-18 14:36:15 +00:00
Trim repetition and historical commentary in developer documentation and agent guidance without changing contracts or behavior. Areas: runtime and ownership, Missions, memory, browser and transcript workflows, settings, repository skills and bundled Design recipes. Preserve current dev contracts, safety boundaries and agent rules. Proposed design changes remain outside this editorial pass. Validated with Design unit and bundle tests, isolated native skill discovery and loading, palette quality, UI typecheck and help tests.
304 lines
24 KiB
Markdown
304 lines
24 KiB
Markdown
# Session content pruning through a V2 plugin
|
|
|
|
## Status and safety boundary
|
|
|
|
A native local plugin entry registers `preview` and `prune`.
|
|
Unsupported or absent plugins produce a localized failure, never a
|
|
legacy API fallback or optimistic deletion. Version labels never gate it; see
|
|
[the audited boundary](SESSION_PRUNING_SAFETY.md).
|
|
|
|
## Integration
|
|
|
|
Pruning edits stored messages through this plugin's own SQLite access and RPC
|
|
methods, the integration OpenCode's maintainers recommend for plugins
|
|
([guidance](https://github.com/anomalyco/opencode/issues/44984#issuecomment-5602099543)).
|
|
|
|
## Modules and request flow
|
|
|
|
`packages/server/src/opencode/session-pruning/`:
|
|
|
|
- `contract.ts`: bounded, strict RPC schemas, no SQL/path/replacement-content inputs.
|
|
- `revision.ts`: browser-safe canonical content hashing.
|
|
- `planner.ts`: completed-assistant checks and technical-part-only selection.
|
|
- `preview-store.ts`: explicit read-only database access, including history before compaction.
|
|
- `media-blob.ts`: the native read form of rows stored with OpenCode's optional media storage (`experimental.media_blob`: `media:<sha256>` refs into `blob`, owned through `blob_ref`). Preview, pruning, prune candidates, navigation windows and assets inline refs, or give the native unavailable placeholder, exactly as native reads do, within the same byte budgets measured on that form. Rows naming no ref pass through untouched; ref-backed rows whose blob tables are missing or unrecognized fail closed as `unsupported_storage`.
|
|
- `database-path.ts`: daemon-side OpenCode data-directory/environment conventions, with an optional path override.
|
|
- `transaction.ts`: synchronous content-only CAS and atomic idempotency receipt.
|
|
- `message-storage.ts`: bounded native message relation for inline content and ordinal `session_part` rows followed by any inline tail; schema inspection selects the stored representation. Headers account for both stores before JSON materialization. Invalid part ownership, type or call identity is refused. Metadata-only rowid/sequence seeks retain their native tables.
|
|
- `claim-fence.ts`: native execution-claim and database-identity checks.
|
|
- `service.ts`: database connection and a fresh `ctx.storage` challenge.
|
|
- `isolated-store.ts`: in-memory-only test adapter; never a live write bypass.
|
|
- `plugin.ts`: native RPC registration and post-commit notifications.
|
|
- `assets-store.ts`, `assets-gallery*.ts`, `assets-conversation.ts`: tool-file assets, the [image gallery](#image-gallery) and the [Images conversation](#images-conversation).
|
|
- `history-ranked-service.ts`: the presence-owned Code Mode `codenomad.session_search` tool and versioned ranked-history RPC; ranking, scope and bounds are specified in [full-history queries](SESSION_HISTORY_QUERIES.md).
|
|
- `desktop-plugin.ts` / `presence.ts`: register RPC while a CodeNomad backend is present; dispose on final close or lease expiry. At load, before following presence, the entry also registers the Design agent and skills that backend leases request (`../design/register.ts`, contract in [OpenCode compatibility](OPENCODE_COMPATIBILITY.md#agent-conventions)).
|
|
The initial effect awaits registration reconciliation before native activation completes. Each successful presence registration acquires a fresh scope incarnation for the Location's `ctx.location.directory`. Targeted preparation observes that incarnation through a nonce-correlated ACK, a live owned backend lease and the scope's presence record. The wire format, quantitative bounds and cleanup ownership are specified in the compatibility contract above; `desktop-plugin-readiness.test.ts` exercises that protocol and the lifecycle seam.
|
|
Closing the native plugin scope stops its follower and joins that follower's admitted readiness writes before releasing its observations. Other live followers retain their registrations and writes; backend shutdown removes its own lease while OpenCode continues independently.
|
|
Readiness files retain their exclusive creator's handle for identity-checked cleanup: backends own requests, followers own ACKs, scope records and reservations. Reconciliation validates its complete bounded snapshot before publishing new ACKs. Scope admission reserves a file under `presence/readiness/<SHA-256 of the exact native directory>/`, counting all entries in that Location's namespace, including crash residues, against a 256-entry scan/refusal bound. Presence leases and other Locations have independent capacity. Root scope proofs retain their nonce-correlated wire paths; cleanup preserves foreign replacements. Targeted cancellation aborts and joins its owned catalogue transport before returning; shared installation claims retain their own lifetimes.
|
|
|
|
`pruning-lifecycle.ts` and `pruning-installation.ts` beside that directory own
|
|
automatic provisioning and backend leases. The server build bundles the payload
|
|
and its dependencies under `dist/plugins/session-pruning/`, copied by both desktop hosts.
|
|
Provisioning and lease placement follow the `OPENCODE_COMPATIBILITY.md` Agent conventions.
|
|
|
|
`WorkspaceManager.desktopPluginsReady(target)` exposes Location-local preparation
|
|
for fixed pruning consumers. The caller authorizes the exact native directory
|
|
before invoking it and supplies its captured connection, cancellation/budget and
|
|
monotone authority/placement assertion. The five-second preparation budget covers
|
|
catalogue activation and fresh presence proof. A result attests observed registration
|
|
for this wait; callers retain adjacent authorization, cancellation, connection and
|
|
deletion-fence checks and issue one useful RPC. The no-argument overload retains
|
|
its startup-marker semantics for existing callers.
|
|
|
|
The fixed brokers capture the published workspace record, native root Location,
|
|
connection object and monotone deletion/display fences before preparation. Session
|
|
descriptors supply session-scoped directories; history pages authorize their selected
|
|
inventory directory, and ranked history prepares each queried directory on every
|
|
continuation. Session and directory ownership are revalidated after preparation and
|
|
after native IO. The navigation outline retains its `outlineLayout` to `outline`
|
|
adaptation exclusively for `rpc.method_not_found`.
|
|
|
|
Pruning, history, navigation and extension assets use a 15-second request budget;
|
|
batch pruning uses 30 seconds. Preparation consumes the remaining budget, with its
|
|
own five-second admission bound. API 4 retains its 19-second route budget and
|
|
eight-second reads. Cancellation and connection assertions sit immediately beside
|
|
the fixed dispatch. Native carrier guards share the pruning request's signal and
|
|
current-placement assertion. Responses retain their connection/ownership fences.
|
|
|
|
Extension assets and API 4 reauthorize the exact enabled package and permission
|
|
after preparation and before publication. Images conversation resolution retains
|
|
its entered deletion fence through the settlement of admitted IO and uses one
|
|
connection throughout Location resolution, preparation, dispatch and Git exclusion.
|
|
API 4 passes its captured outer connection and current grant into that resolver.
|
|
|
|
Pending recovery prepares only its established, authorized execution-host bootstrap.
|
|
Candidate ownership is admitted before preparation and revalidated afterward; the
|
|
fixed snapshot keeps its loaded-only candidate semantics. Root identity, compaction
|
|
admission, cancellation and connection are checked beside its single HTTP dispatch.
|
|
The route preserves per-directory non-authoritative errors and retained queues.
|
|
|
|
The UI offers individual, per-message, group and session cleanup entry points. It
|
|
fetches the native message, resolves selected tools by ID and reasoning by a unique
|
|
content/time match, then sends indexes plus a canonical SHA-256 content revision.
|
|
Ambiguous reasoning or stale content fails closed. It never submits replacement text.
|
|
The plugin compares that revision with the reconstructed native read form (`message-storage.ts`, `media-blob.ts`),
|
|
deletes selected split rows and removes selected entries from the inline tail under the same write lock.
|
|
Retained split rows keep their original ordinals and JSON bytes; native readers order ordinals without requiring contiguous values.
|
|
The envelope's exact-data CAS, part deletion and receipt commit together. In the same transaction it releases the
|
|
`blob_ref` rows of refs the removed parts alone named, deleting blobs no ref keeps,
|
|
as native content updates do.
|
|
Selections allow up to 100,001 unique indices (0-100,000), within a 1 MiB broker
|
|
body. Each message must fit the 16 MiB stored and serialized native-read budgets,
|
|
with or without media refs; larger messages fail closed.
|
|
|
|
`POST /api/workspaces/:id/session-pruning/{preview,prune}` validates workspace
|
|
ownership from the native session location and participates in the worktree deletion
|
|
fence. It pins `codenomad.session-pruning`, the method and runtime location; the
|
|
instance proxy blocks arbitrary RPC. Server authentication middleware protects
|
|
these routes.
|
|
|
|
Ordinary single-message and batch pruning apply the shared Mission schedule input
|
|
guard before native content writes. Technical carrier sessions stay read-only
|
|
through these brokers; their preview and history reads remain available.
|
|
|
|
After commit the caller re-reads native history, never a synthesized remainder.
|
|
`rpc.codenomad.session-pruning.pruned` contains session/message IDs and the content
|
|
revision. The UI invalidates both message-load authority and its native SDK transcript,
|
|
including pending rotations, then coalesces active-session reloads. Late reads cannot
|
|
overwrite a newer pruning invalidation. Reconnect uses authoritative reconciliation.
|
|
Other clients may display cached history until they reload.
|
|
|
|
The receipt and content change commit together; an identical retry returns the same
|
|
receipt and re-emits the event. Receipts contain no deleted content. Event delivery
|
|
is best-effort: failure cannot roll back a committed prune. A timed-out caller must
|
|
re-read or retry the **same** request, never assume no mutation occurred.
|
|
|
|
## Image gallery
|
|
|
|
Fixed methods recompute every answer from one session's transcript inside a
|
|
`BEGIN … ROLLBACK` read, through `withHistoryDatabase` and `ownedSession`: no cache,
|
|
index, copy, file access or `kv` lookup beyond the storage challenge. Every query
|
|
stops before the revert boundary (`id < revert.messageID`).
|
|
|
|
- `images({ sessionID, cursor?, prompts? })` lists, newest first, one `image` entry
|
|
per image file (`image/png|jpeg|gif|webp|avif`) of a `read` tool part and one
|
|
`generation` entry per `experimental_image_generate` part, in part and file order. A page holds
|
|
at most 32 assistant messages, 32 MiB of stored JSON (each row size-checked
|
|
before it loads; a row over 16 MiB gives `blocked: limit`), 64 entries and 1 MiB
|
|
serialized. Every admitted row is emitted before the cursor advances: a row the
|
|
byte budget refuses becomes the cursor at offset 0; a full page points at the next
|
|
entry. Cursors bind the scope and boundary (`blocked: conflict` otherwise) and
|
|
carry `through`, the highest assistant or synthetic `seq` of the first page.
|
|
Media refs list from blob sizes only, as `assets` does.
|
|
- `imageRead({ sessionID, target })` is `assetRead`'s byte read behind a membership
|
|
check in the same bounded read: anything but an image file of a `read` part gives
|
|
`blocked: not-gallery`; a row over 16 MiB gives `blocked: limit`. Decode bounds
|
|
belong to the caller's channel.
|
|
- `imageReference({ sessionID, target })` takes an image target or a generation
|
|
target `{ messageID, part, digest: SHA-256(part id, NUL, jobID) }` and returns
|
|
`{ path }` relative to the location, or `refused` with `unverifiable`, `outside`,
|
|
`changed`, `not-gallery` or `missing`. A target is never substituted.
|
|
|
|
Generation annotations (`generationKey`, `parentKeys`, `otherRefs`) appear only on
|
|
proven images; other images are `origin: "unverifiable"` and keep their pixels.
|
|
The proof covers the window rows, up to 32 older assistant rows within 32 MiB, and
|
|
one scan of at most 4,096 synthetic rows from the lowest covered `seq` to `through`,
|
|
decoded by SQLite only for rows of at most 64 KiB. A read of file `F` over
|
|
`[time.ran, time.completed]` is proven when the notice scan is complete, no
|
|
undecoded row is dated at or before the read's end, the scan covers the item's whole
|
|
range (its message and the 31 before it, within 32 MiB, unless the session starts
|
|
sooner; a range an oversized row or the scan budget cuts proves nothing), exactly one
|
|
`experimental_image_generate` call that may write `F` in that range started before that end
|
|
(a call whose single decoded notice reports `error` before that call's start does not count:
|
|
OpenCode delivers a failed job's notice once its run has settled, and the run's last step is the
|
|
write, so a failed attempt followed by a retry under the same name stays provable; a cancelled
|
|
call still counts, since its notice can precede the end of an interrupted write), it
|
|
completed with a `jobID` before the read began, its single
|
|
decoded notice names `F` between the call's start and the read's start, and no
|
|
other completed notice names `F` from then until the read's end. Parents of a proven
|
|
call follow the same rule over the generation's own interval. Only a `completed`
|
|
`experimental_image_generate` part with `metadata.path` names the file it writes (that stem);
|
|
every other `experimental_image_generate` part (streaming, running, errored or completed without
|
|
`metadata.path`) may write any file, so it leaves every proof and reference it could
|
|
affect unverifiable. Relative read names and parent references resolve lexically against the session directory in the daemon's
|
|
path style (Windows without case); one outside the location has no label and no
|
|
reference. Generation states are the native part status and, when established, the
|
|
outcome of its unique decoded notice (`completed`, `error`, `cancelled`), else
|
|
`unknown`. `prompts: true` adds `prompt` (8 KiB of UTF-8, then `promptTruncated`)
|
|
and `model` to generations and proven images only in a root session marked
|
|
`codenomad.companion: images`; the host grants it. Entries never carry paths,
|
|
directories, job, notification or call IDs, or notice and error text.
|
|
|
|
References also require no later write: the newest 32 assistant rows within 32 MiB
|
|
must reach the item, with a complete notice scan, no undecoded row dated from the
|
|
read's start (a generation's notice time), no call that may write any file started
|
|
from then (`unverifiable`), and no call to the same stem started from then or other
|
|
notice naming the file after it (`changed`). A generation target is also refused as
|
|
`unverifiable` when the scan cuts its range, or while another call that may write
|
|
its file, started before its notice (or at an unknown time) in the item's range or
|
|
later, lacks a completion time or a decoded notice of its own.
|
|
|
|
Consumed contracts: `read` tool `state.content` files, `experimental_image_generate` parts
|
|
(`state.status`, `state.input.prompt|images`, `state.metadata.path` as the stem, `jobID`, `model`,
|
|
`time.ran|completed`) and its synthetic notices (`metadata.source|jobID|state|path`,
|
|
text `Saved image: <file> (`). A change leaves generations `unknown` and images
|
|
unverifiable.
|
|
|
|
## Images conversation
|
|
|
|
`imagesConversation({ mode, title?, agent?, model? })` runs for the calling location.
|
|
The lookup reads `session_v2` with `ownedSession`'s location predicate, root and
|
|
unarchived sessions whose `metadata."codenomad.companion"` is `images`, newest by
|
|
`time_created DESC, id DESC`, over at most 20,000 roots: `found`, `absent` only after
|
|
a complete scan, else `unknown`. Every mode also answers `unknown` when the database
|
|
or its storage challenge cannot be read or fails, and creates nothing; cancellation
|
|
still rejects. `open` repeats the lookup under a process-wide lock
|
|
(global symbol, keyed by database file, project, storage directory and workspace)
|
|
and creates through `ctx.session.create` only when `absent`; `new` always creates.
|
|
The lock serializes the callers of one OpenCode process; two processes sharing a
|
|
database can each create one, and the lookup then picks the newest.
|
|
|
|
`POST /api/workspaces/:id/images-conversation` (`server/routes/images-conversation.ts`)
|
|
brokers it for `{ directory, mode, title?, agent?, model? }`: the workspace manager
|
|
authorizes the directory and the worktree deletion fence is entered before any OpenCode
|
|
request (reading a Location boots its services); only then is the Location read and
|
|
checked again. RPC ID and method are pinned, and
|
|
a missing method, malformed answer or timeout is `blocked: unavailable`, never a
|
|
fallback. In the UI (`ui/src/stores/images-conversation.ts`), `resolveImagesConversation`
|
|
refreshes the current location's agent catalog and sends the localized title,
|
|
`agent: "design"` only when the catalog lists it, and the model `getDefaultModel`
|
|
resolves for that agent; the plugin applies them only when it creates. The agent list
|
|
counts only when it was read for that Location (`agentCatalogLocations`); otherwise the
|
|
answer is `unknown` and nothing is created.
|
|
`showImagesConversation` then hydrates and selects a found conversation and requests
|
|
composer focus, so a caller can answer before navigating; nothing is ever sent. After a successful `open`
|
|
or `new` the route excludes the Location's `.codenomad/design/` (`workspaces/design-git-exclude.ts`)
|
|
and answers `gitExclusion`: the execution host's Git (`wsl.exe --distribution <owned>
|
|
--exec git` for WSL, in the C locale) resolves `rev-parse --show-prefix --git-path info/exclude`,
|
|
the path is translated for host access, and `appendCodenomadGitExclude` appends the
|
|
rule under `# codenomad` without rewriting existing bytes, never `.gitignore`. At a
|
|
worktree top the rule is `.codenomad/design/`; a nested Location gets
|
|
`/<prefix>.codenomad/design/` with glob characters escaped, since slash-containing
|
|
patterns anchor at the worktree top (the shared `info/exclude` applies at each
|
|
worktree's top). Existing lines compare as Git reads them: a final `\r` and unescaped
|
|
trailing spaces are ignored, leading whitespace is part of the pattern. Only
|
|
Git's explicit "not a git repository (or any …)" answer is `not-repository`; Git
|
|
missing, an unfollowable `.git` pointer, a failed translation or a location removed
|
|
meanwhile write nothing and give `unavailable`.
|
|
|
|
## Version and loading notes
|
|
|
|
`history` and `pruneBatch` extend this plugin; see
|
|
[full-history queries](SESSION_HISTORY_QUERIES.md).
|
|
|
|
- The broker uses the published low-level `rpc.call` transport. The plugin builds through `packages/server/scripts/native-plugin-build.mjs`, which bundles application dependencies and leaves `effect` and `@opencode/plugin` to the daemon (never embed them); startup installs nothing from npm, and discovery is never gated on an exact runtime version. The native test runs the stable client against an explicitly supplied isolated official runtime, including RPC, custom events and the guarded UI proxy.
|
|
- The plugin contract lacks `ctx.session.message` and `ctx.db`; the plugin opens its own SQLite connection.
|
|
- SQLite unit tests use Node's `node:sqlite`; the native test also exercises that
|
|
driver **inside the official compiled runtime**.
|
|
- The module has a packable manifest with `index.ts` and `./rpc` entrypoints.
|
|
Local configuration must target the **directory**, never `plugin.ts`.
|
|
- An automatically provisioned entry in the global OpenCode `plugins` directory is
|
|
discovered for all projects. Its small lifecycle module stays loaded; pruning
|
|
RPCs exist only while CodeNomad is present. Pruning commands never upgrade or
|
|
restart the daemon.
|
|
|
|
## Coordination and validation coverage
|
|
|
|
Locking and claim checks: [the audited boundary](SESSION_PRUNING_SAFETY.md).
|
|
|
|
The native fixture covers real preview/prune RPC, active-generation refusal, both
|
|
prompt/transaction orderings, retry receipts, two event subscribers, next primary
|
|
request payload, fork isolation, pre-compaction deletion without summary changes,
|
|
server restart, and a tool file result read through navigation, assets and pruning
|
|
in whichever form the runtime stores it (with `experimental.media_blob`, a blob ref
|
|
whose ref and blob the prune releases), the Images conversation (one creation for
|
|
concurrent opens from two clients, a newer marked child, deletion, another location,
|
|
restart) and gallery membership on that stored tool file. Unit tests cover rollback
|
|
and client projection/read races; the gallery tests replay proof mutations on
|
|
Images transcripts with and without media blobs
|
|
(`assets-gallery-fixture.json`).
|
|
`scripts/test-design-companion-native.mjs` drives an Images conversation end to end with a
|
|
scripted chat model and a fake image model on the shipped bundle: two series of read images
|
|
under `.codenomad/design/`, proven with prompt, lineage and pixels, then a completed
|
|
generation without read-back whose reference is sent and read more than 32 assistant
|
|
messages later (pixels, unverifiable origin). With `--ui <Design ZIP>` the same request is
|
|
prefilled from the installed Design extension through the real right panel, API 4 routes
|
|
and composer, sent, and its relative Markdown image is checked in the real transcript.
|
|
The mock's usage numbers are synthetic; never cite them as provider token savings.
|
|
|
|
Unvalidated: native WSL runs, two desktop windows/scroll,
|
|
provider-specific continuation state and budgets, and interactive bulk
|
|
progress/cancel. Never describe these as passed on the strength of HTTP or
|
|
mocked cache tests.
|
|
|
|
## Checks
|
|
|
|
```powershell
|
|
node --import tsx --test packages/server/src/opencode/session-pruning/*.test.ts packages/server/src/server/routes/session-pruning.test.ts packages/server/src/server/__tests__/instance-proxy.test.ts
|
|
node --import tsx --test packages/server/src/server/routes/images-conversation.test.ts packages/server/src/workspaces/design-git-exclude.test.ts # CODENOMAD_TEST_WSL_DISTRO=<distro> adds the real WSL case
|
|
node --import tsx --test tests/browser/images-conversation.test.ts # from packages/ui
|
|
node --import tsx --conditions=browser --test --test-force-exit packages/ui/src/stores/session-actions.test.ts packages/ui/src/stores/session-pruning-events.test.ts packages/ui/src/stores/session-pruning-projection.test.ts packages/ui/src/stores/opencode-data.test.ts
|
|
node scripts/test-session-pruning-native.mjs C:/isolated-install/opencode2.exe
|
|
node --import tsx scripts/test-pruning-location-ready-native.mjs C:/isolated-install/opencode2.exe
|
|
node scripts/test-design-companion-native.mjs C:/isolated-install/opencode2.exe # gallery end to end, scripted chat and image models
|
|
node scripts/test-design-companion-native.mjs C:/isolated-install/opencode2.exe --ui C:/abs/neuralnomads.design-1.0.0.zip # plus the Design panel, composer and transcript
|
|
npm run typecheck --workspace @neuralnomads/codenomad
|
|
npm run typecheck --workspace @codenomad/ui
|
|
npm run build --workspace @codenomad/ui
|
|
```
|
|
|
|
UI store tests retain application timers; `--test-force-exit` terminates those after
|
|
the assertions finish. The separate native test creates its own DB/config/home,
|
|
starts `serve --port 0` directly (never service discovery), supplies a local provider,
|
|
and terminates only its own child. It retains its synthetic fixture/logs for inspection.
|
|
An optional third argument tests an independently installed plugin package directory.
|
|
The pruning driver awaits the exact `codenomad-session-pruning` active catalogue
|
|
entry in each captured Location, with that client's location options and a 60-second
|
|
abort deadline, before its first useful RPC and after daemon restart. The shared
|
|
`scripts/native-pruning-ready.mjs` keeps missing/failed activation diagnostics and
|
|
the driver's bounded polling checks. Presence withdrawal/reopening has its own
|
|
explicit lifecycle assertions in the bundled run.
|
|
|
|
## Agent conventions
|
|
|
|
- Session pruning is a narrow V2 plugin/RPC exception under `packages/server/src/opencode/session-pruning/`. Bundle it with the shared server for both desktop hosts and provision through normal native plugin discovery. RPC registrations follow backend presence; clean shutdown removes that backend's lease and crashes expire. Loading never deletes content. Deletion occurs only on an explicit pruning request, never behind an enable-write switch or version gate. Keep generic RPC proxy access closed. Writes validate actual storage, a fresh daemon-storage identity challenge and the native durable execution claim inside a synchronous SQLite transaction. Run isolated native concurrency/payload and client-cache regressions; tests must never target the shared daemon or a user's database.
|