mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-10-29 18:44:34 +00:00
02c3ab21dd
An audit of everything on main since v0.9.23 turned up five places where real detail had been used where an invented value would have done. None of them is a credential and none identifies a customer, but a public repository is the wrong home for any of it. perf/README.md cited an absolute path into a private audit workspace on a contributor's own machine, naming their home directory, the workspace's naming scheme and a run timestamp accurate to the second, as "evidence" no reader can open. The number stays; the path goes, and the text now says plainly that the artifact is not published. The same file described the measurement corpus as the founder's 35 GB of session history, which is a personal fact about one identifiable person, so it is now described by what it is rather than by whose it is. The remaining three are fixtures: a private product name as a working directory, the maintainer's real operating-system username inside the very tests that assert such a path is redacted, and their two machine names in the sharing tests. Using the real values proved nothing the invented ones do not, and the redaction tests read better when the string being scrubbed is obviously synthetic.
44 lines
3.2 KiB
Markdown
44 lines
3.2 KiB
Markdown
# CodeBurn performance harness (Phase 0)
|
||
|
||
Measure → change → measure. This directory is the hill: one command per metric, pinned baselines, and an append-only iteration log.
|
||
|
||
It extends the [release-acceptance ledger](../docs/release-acceptance/README.md) from #1164. It does **not** replace that runner. `scripts/release-acceptance/run.mjs` stays the SHA/package gate. These scripts emit the `timings.csv` template shape and wait-path numbers a hostile reviewer can re-run in one command.
|
||
|
||
## What is measured
|
||
|
||
| Metric | Command | What the number is | What it is not |
|
||
|---|---|---|---|
|
||
| Cold start (wait-path) | `run-metric.mjs --metric cold-start-cli` | One-shot `status --format menubar-json` argv Desktop/Menu Bar already spawn | Installed window show / flame in the menu bar |
|
||
| Session parse | `--metric session-parse` | Cold parse of the synthetic fixture; MB/s and ms | A large real corpus |
|
||
| Incremental re-parse | `--metric incremental-reparse` | Append two JSONL lines on the same inode, then re-run | Rewrite / inode change |
|
||
| Period switch | `--metric period-switch` | Warm `serve --stdio` 7D/30D p50/p95 | Installed Desktop 250 ms p95 on the live corpus |
|
||
| Dock/TUI interactions | `--metric dock-tui-proxy` | Overview-refresh and view-switch wait path | Capacity Dock hover (native, never spawns this argv) or TTY TUI keys |
|
||
| Memory | `--metric memory` | RSS after cold load; optional `--idle-ms 3600000` | Packaged app RSS |
|
||
|
||
Historical receipt, not this fixture: installed Desktop 0.9.21, live corpus, 2026-08-27, 7D useful summary **18286.98 ms** after 30 s idle. Spec target: **250 ms p95** ready summary, zero raw reads on unchanged generation. That receipt came from an installed-app audit run that is not published, so treat the number as historical context rather than something you can re-derive from this repo.
|
||
|
||
## One-command reproduction
|
||
|
||
```bash
|
||
HOME_DIR=$(mktemp -d /tmp/codeburn-perf-XXXX)
|
||
node scripts/perf/gen-fixture.mjs --home "$HOME_DIR" --target-mb 30
|
||
node scripts/perf/run-metric.mjs --metric all --home "$HOME_DIR"
|
||
```
|
||
|
||
Or `npm run perf:all -- --home "$HOME_DIR"`. To pin a new phase, paste the run's `summary.json` numbers into `perf/BASELINES.md` by hand — pinning happens once per phase and does not need a script.
|
||
|
||
The fixture is synthetic (fake `/work/...` paths, lorem tool payloads). Do not copy real session files into the repo or a PR. Generated bytes live under the isolated HOME, which is gitignored.
|
||
|
||
## Isolation
|
||
|
||
Matches `scripts/upgrade-path/run.mjs` `cliEnv()`: isolated `HOME` / `USERPROFILE` / `APPDATA` / `LOCALAPPDATA` / `CODEBURN_CACHE_DIR`. The generator refuses to write under the real home.
|
||
|
||
## Iteration contract
|
||
|
||
1. Attack one baseline row.
|
||
2. Branch `perf/<metric>-<short-name>` from current main.
|
||
3. One change. Measure with this harness.
|
||
4. Draft PR with before, after, command, fixture. Never merge from this lane.
|
||
5. Append `perf/ITERATION-LOG.md` (kept or reverted).
|
||
|
||
Product files this harness must not touch: `src/parser.ts`, `src/session-cache.ts`, `src/dashboard.tsx`, `app/electron/main.ts`, `app/electron/cli.ts`, `mac/Sources/CodeBurnMenubar/**`. Those surfaces have in-flight or just-landed quality work (PRs 1159–1171, open #1149/#1062/#940).
|