mirror of
https://github.com/NeuralNomadsAI/CodeNomad.git
synced 2026-10-22 14:22:22 +00:00
debbdce33f
## Summary Adds a fifth Mission template, **Compare two models** (`compare`). It runs the same bounded task on two different models and ends with a blind verdict. - **Creation:** the user picks **model A** and **model B** in the main creation form. Both are required and must name different models (provider + model id; a reasoning variant alone is not a different model). This is enforced on every creation path: HTTP, prepare, recurrence create and agent-started missions. - **Roles:** `model-a`, `model-b` and `comparator`. - A and B share one instruction text and report contract. - The coordinator writes the task, inputs and criteria once, and gives both byte-identical briefs. - The server refuses a run task whose brief differs from its sibling's, or whose per-task execution would change its role's model. - **Blind judge, always on:** the comparator works on neutral X/Y labels. - It reports one versioned `comparison` artifact: the A/B task references, per-criterion scores for both runs, sourced observations, and a verdict `a | b | equivalent | inconclusive`. - Completion requires both runs and that comparison. - **Judge model:** frozen once at creation, like every other role. - An explicit judge is kept; otherwise the judge copies the coordinator's effective model. When that model is unknown, both stay on the native default. - Replays and retries therefore reuse the stored profile. - The editor warns when the judge's model equals A or B. - **Readers:** the verdict, both model names, a score table and evidence by path. Hashes stay under Technical details, for current and archived recurring passages. The template came out of evaluating Harness2's comparison loop. It deliberately has **no** lesson, human approval, budget or duration handling, and leaves the Wayfinder human gate unchanged. Up to five models, per-run worktrees and launching from the composer's model picker (OpenChamber-style multi-run) are left for later. `AGENTS.md` records the single editor exception: the A/B selectors sit outside the collapsed Options disclosure. All ten locales are updated. ## Size and upkeep 62 files, +837/-96: | Part | Lines | |---|---| | Product code (about 16 files are one-line template-list hooks) | +280/-57 | | Locales (13 keys × 10) | +131 | | Tests | +409 | | Docs | +17 | `missions/control.ts` stays oversized (~1,500 lines). ## Validation - Full server suite: 2,426 tests pass, 7 existing skips. Root and server typechecks, the server build and the plugin build pass. - UI: 45 unit tests and 36 browser tests pass. - The native trajectory fixture, extended with a compare case, passes on OpenCode 2.0.31. It checks two distinct models, a one-newline brief difference refused with the journal unchanged, a real blind judgment and a completed mission. The native contract and recurring Q fixtures also pass. - Regressions cover: - distinct-model refusals on every creation path; - brief and override refusals; - artifact validation and the completion policy; - judge freezing at creation, revision replay, and retry after a coordinator model change; - editor inheritance and readable evidence. - Independent review rounds ended with no P0-P3 findings. - The editor and reader were checked from screenshots of the real Solid components in the browser harness, not in a packaged app. |
||
|---|---|---|
| .. | ||
| fixtures | ||
| missions-child-environment | ||
| recurring-simple-native | ||
| winget | ||
| benchmark-session-history.mjs | ||
| build-help.mjs | ||
| bump-version.js | ||
| compare-user-html-rendering.mjs | ||
| desktop-server-resources.cjs | ||
| desktop-server-resources.test.cjs | ||
| native-fixture-diagnostics.mjs | ||
| native-fixture-diagnostics.test.mjs | ||
| native-fixture-guards.mjs | ||
| native-fixture-guards.test.mjs | ||
| native-product-fixture.mjs | ||
| prepare-node-runtime.cjs | ||
| prepare-node-runtime.test.cjs | ||
| preview-git-history.mjs | ||
| profile-outline-projection.mjs | ||
| profile-outline-responsiveness.mts | ||
| profile-streaming-responsiveness.mjs | ||
| profile-ui-motion.mjs | ||
| run-session-pruning-native.mjs | ||
| smoke-packaged-resources.cjs | ||
| test-attachment-picker-desktop.mjs | ||
| test-automation-native.mjs | ||
| test-blank-session-cleanup-native.mjs | ||
| test-compaction-isolation-native.mjs | ||
| test-electron-owner-close.mjs | ||
| test-git-degraded-native.mjs | ||
| test-git-init-native.mjs | ||
| test-help-browser.mjs | ||
| test-help.mjs | ||
| test-missions-native-contract.mjs | ||
| test-missions-native-family-control.mjs | ||
| test-missions-native-trajectory.mjs | ||
| test-missions-native.mjs | ||
| test-native-event-relay.mjs | ||
| test-native-worktree-management.mjs | ||
| test-opencode-history-migration.mjs | ||
| test-opencode-install-native.mjs | ||
| test-opencode-legacy-replacement-native.mjs | ||
| test-opencode-location-native.mjs | ||
| test-opencode-proxy-native.mjs | ||
| test-opencode-setup-desktop.mjs | ||
| test-opencode-setup-wsl.mjs | ||
| test-opencode-upgrade-native.mjs | ||
| test-output-limit-native.mjs | ||
| test-panel-extension-native.mjs | ||
| test-pending-snapshot-native.mjs | ||
| test-permission-receipts-native.mjs | ||
| test-plugin-packages-native.mjs | ||
| test-plugin-runtime-native.mjs | ||
| test-promise-tool-failure-native.mjs | ||
| test-prompt-skills-native.mjs | ||
| test-provider-account-selection-native.mjs | ||
| test-provider-accounts-native.mjs | ||
| test-provider-usage-native.mjs | ||
| test-recurring-simple-native.mjs | ||
| test-release-workspace-version.mjs | ||
| test-render-cost-audit.mjs | ||
| test-service-usage-native.mjs | ||
| test-session-aside-native.mjs | ||
| test-session-environment-native.mjs | ||
| test-session-fork-native.mjs | ||
| test-session-history-native.mjs | ||
| test-session-interrupt-native.mjs | ||
| test-session-navigation-native.mjs | ||
| test-session-pruning-native.mjs | ||
| test-session-pruning-ui.mjs | ||
| test-subagent-depth-settings-native.mjs | ||
| test-tauri-f6.ps1 | ||
| test-tauri-input-deadlock.mjs | ||
| test-tauri-macos-window-result.test.mjs | ||
| test-tauri-macos-window.mjs | ||
| test-tauri-runner-refcount.mjs | ||
| test-tauri-view-menu.mjs | ||
| test-temporary-workspaces-native.mjs | ||
| test-websearch-settings-native.mjs | ||
| test-worktree-families-native.mjs | ||
| verify-bundled-node.cjs | ||