vrr/ouroboros
2
0
Fork 0
mirror of https://github.com/razzant/ouroboros.git synced 2026-10-19 02:02:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7

release: Ouroboros v6.26.0-rc.1 — systemic hardening: immune system, custody, memory integrity, native multimodal chat

Browse source 
Owner-approved big release (plan: большой_релиз_v6.26.0). Workstreams:

- WS-A lint gate: ruff F-rules step in CI quick-test; fixed the real F823
  NameError class (supervisor/events.py utc_now_iso shadowing); make lint/health.
- WS-B memory integrity: atomic dialogue_blocks with corrupt-quarantine
  (.corrupt-<ts>.bak + memory_store_corrupt event); honest scratchpad journal
  (block_append_failed, corrupt storage renders as corruption not "(empty)");
  full Pattern Register window (16K cap vs 3K cut); merge-aware scratchpad
  consolidation under the sidecar lock; backlog writes through locked helpers;
  chat omission notes ("[N older unconsolidated messages omitted]").
- WS-C provider SSOT: provider registry (prefixes/credentials/resolution) in
  provider_models.py replacing 5 duplicated knowledge sites; credential-aware
  consolidation model; pricing empty-fetch retry; direct-route correctness
  (o-series max_completion_tokens, reasoning_effort, Anthropic error bodies,
  per-request timeouts on cached clients); deep/plan review budgets reserve
  output headroom inside the 1M window (min(SSOT, window − output − margin)).
- WS-D races/state: utils.update_json_locked (locked RMW, loud TimeoutError);
  task_results merge under per-file lock (cancel-latch safe); update_state
  migration for owner binding, evolution counters, budget_line, post-task
  activation; queue-lock coverage (enqueue, timeouts, worker health, task_done,
  snapshot); visible supervisor death (supervisor_error + owner notice + ghost
  consciousness/chat-agent cleanup on re-init); chat.jsonl rotation via
  os.replace under the append lock; unbounded _outbox removed; host-service
  port probe + actual-port panic sweep; skill lifecycle lane deadline
  (OUROBOROS_SKILL_LIFECYCLE_TIMEOUT_SEC, dedupe-leak-safe); consilium
  force-plan structured flag (plan_review_aggregate from the FULL result);
  atomic writers (update-intent, repo-manifest, metrics cache, post-task pair);
  review_state.save_state raises on lock timeout (advisory ledger honesty);
  apply_pending_request no-ops while a campaign is active;
  api_update_apply kills workers only AFTER update validation, with respawn
  on aborted checkout and "interrupted" terminal status.
- WS-E immune hardening: triad anti-refusal coverage contract (empty array
  needs NO_FINDINGS sentinel or bare-[] body; refusal prose with [] is
  parse_failure and never enters quorum); BIBLE P3 "Owner-chosen enforcement,
  loud advisory" bound + CHECKLISTS sync; every advisory pass-through of a
  blocking signal writes review_advisory_override + persistent
  advisory_overrides (surfaced by review_status); ToolEntry.mutates_worktree
  with dispatcher-level before/after worktree diff invalidation (covers error
  paths; read-only runs no longer invalidate; redundant manual calls removed);
  scope review fails closed without its checklist; advisory final write is a
  locked re-read merge; synthesis severity defaults to critical with WARNING
  fallbacks; find removed from SAFE_SHELL_COMMANDS; anti-thrashing state
  survives advisory criticals; preflight env scrubs secret-class variables;
  .git/index.lock age gate in worker startup checks.
- WS-F security: file-browser symlink containment on the RESOLVED path across
  all endpoints (out-of-root symlink targets are listed but inert; tests
  rewritten to the new contract deliberately); HMAC-signed session cookies
  (server-side persisted key, 30-day TTL, Secure on TLS) replacing the
  permanent password-derived cookie; password-class settings mask to a
  constant placeholder; conservative SSRF guard for the MAIN agent (link-local
  /cloud-metadata only, LAN stays reachable, per-request route re-validation);
  ClawHub zip-slip hardening (":"/backslash segments rejected + post-join
  containment) and lazy no-proxy OuroborosHub opener; single-execution
  signature dispatch for extension handlers (no TypeError re-run after side
  effects); onboarding postMessage origin checks; SHA256-pinned
  python-standalone download with pipefail; payload-resident dependency
  fingerprints only corroborate durable deps.json; skill payload re-hash
  immediately before spawn (TOCTOU narrowing).
- WS-G process custody: ouroboros/process_custody.py — spawn_supervised
  chokepoint + durable data/state/process_ledger.jsonl (pid, pgid,
  fingerprint{start_time, cmd_sha256}, purpose, scope task|session|daemon,
  owner_task, session_id); platform_layer.process_start_time primitive;
  startup + periodic reaper killing ONLY strict-fingerprint matches from dead
  generations/tasks (never by command-line class); migrations: services
  (the orphan hole), workspace executor + local model + extension companions
  (ledger write-through), worker_pids (write-through; legacy path retained);
  parent lifelines (ppid watchdog, group-suicide only as group leader) in
  worker_main, extension runner, Claude readonly child; conformance test
  pinning the Popen allowlist; ARCHITECTURE/DEVELOPMENT/CHECKLISTS entries.
- WS-H native multimodal chat: supports_vision capability map (static
  prefixes + OpenRouter /models input_modalities overlay); web chat uploads
  ride the WS frame as structured attachments (additive ChatInbound field)
  and image uploads become NATIVE image blocks via the existing Path B;
  browser screenshots inject natively for vision models via the multipart
  user-merge (tool result stays a string; file persisted under
  data/uploads/screenshots for re-view); K=3 newest-image eviction with
  caption placeholders carrying the vlm_query re-view path; image-aware token
  estimates (fixed ~1.1K-token equivalent instead of base64 length, fixing
  permanent emergency-compaction wedges); compaction renders images as
  captions (no base64 into the summarizer); GigaChat/local lanes emit explicit
  "[image omitted: model has no vision]"; internal _caption/_source_path
  metadata stripped from provider payloads.
- WS-I housekeeping (partial): SETTLED_STATUSES SSOT (+ cycle-safe mirror pin);
  owner_inject.py renamed to owner_mailbox.py; version-neutral envelope
  wording; files.py import-block cleanup. Remaining WS-I/WS-J/WS-K items are
  deferred with the owner's context-budget priority on review+release.

Review notes: triad+scope ran via scripts/run_external_review.py on the core
pack across 3 rounds to convergence (scope responded=PASS each round; round-2
criticals fixed: update_apply kill-order + respawn, lifecycle dedupe leak on
lane timeout, OUROBOROS_MAX_ROUNDS hot-reload + docs, toggle_evolution
NameError, async Anthropic timeout forwarding, supports_vision local check,
budget-update lock visibility, ChatInbound additive attachment contract,
file-browser doc sync). The FULL combined diff exceeds every triad model's
context window (~1.59M tokens > 1.05M) — reviewed in packs; remaining
cross-pack findings were verified as slicing artifacts. Adversarial critics
(GPT; Gemini/Opus rounds) ran on the working tree. Deliberate tradeoffs:
metadata-based eviction captions (no light-LLM call in the hot path);
worker ledger records use live-cmdline fingerprints with a synthetic-arg
fallback only where the OS offers no cmdline.
This commit is contained in:
Ouroboros 2026-06-10 15:28:34 +03:00
parent 0088373be7
commit ae53779ccd
187 changed files with 3159 additions and 857 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.github/workflows/ci.yml vendored
View file

@ -61,7 +61,9 @@ jobs:
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install pytest
pip install pytest ruff
- name: Lint (deterministic F-rule gate — catches the NameError-under-except class)
run: python -m ruff check . --select F
- name: Run tests
run: python -m pytest tests/ -q --tb=short
- name: Guard extracted transport imports stay out of core

19
BIBLE.md
View file

@ -203,10 +203,13 @@ may not weaken it.
### Components
1. **Multi-model blocking review** at every commit. Diff reviewers
score the staged diff against the checklists in
[docs/CHECKLISTS.md](docs/CHECKLISTS.md); a blocking scope reviewer
examines goal / intent / coupling using broader repository context.
1. **Multi-model review** at every commit. Diff reviewers score the
staged diff against the checklists in
[docs/CHECKLISTS.md](docs/CHECKLISTS.md); a scope reviewer examines
goal / intent / coupling using broader repository context. The gate
is blocking under `blocking` enforcement; under owner-chosen
`advisory` enforcement it still runs in full and every decision that
blocking would have stopped is loudly and durably recorded.
2. **Advisory pre-review**: a cheaper preflight on the same snapshot,
mandatory before the blocking review; staleness-aware (any
worktree-mutating tool invalidates the snapshot).
@ -257,6 +260,14 @@ following bounds are constitutional:
[docs/CHECKLISTS.md](docs/CHECKLISTS.md), is a named exception scoped
to cosmetic changes and release-artifact metadata. Every bypass is
durably audited. Silent bypass is forbidden.
- **Owner-chosen enforcement, loud advisory.** The owner selects review
enforcement (`blocking` or `advisory`). Advisory enforcement is
legitimate ONLY while every decision that blocking enforcement would
have stopped — critical findings, quorum failure, infrastructure
failure, missing advisory provider — leaves a loud, durable,
owner-visible trace at the moment it happens. Silent advisory is
forbidden; an advisory mode that hides what it waved through is a
weakened immune system, not a configured one.
- **Self-modification gate.** Any change to these bounds (scope floor,
context floor, bypass rules, durable-memory permanence) is itself a
constitutional change and requires plan review. Changes that

10
MagicMock/mock.drive_root.__truediv__().__truediv__()/4419367024 Normal file
View file

@ -0,0 +1,10 @@
{
"count": 1,
"recent": [
{
"ts": "2026-06-10T12:38:44.359372+00:00",
"block_reason": "review_quorum",
"message_head": "⚠️ REVIEW_BLOCKED: Only 1 of 1 review models responded successfully (minimum 2 required). Unavailable/failed: .\nRetry the commit — transient model failures usually resolve quickly."
}
]
}

10
MagicMock/mock.drive_root.__truediv__().__truediv__()/4712967760 Normal file
View file

@ -0,0 +1,10 @@
{
"count": 1,
"recent": [
{
"ts": "2026-06-10T12:30:14.271758+00:00",
"block_reason": "review_quorum",
"message_head": "⚠️ REVIEW_BLOCKED: Only 1 of 1 review models responded successfully (minimum 2 required). Unavailable/failed: .\nRetry the commit — transient model failures usually resolve quickly."
}
]
}

11
Makefile
View file

@ -11,12 +11,17 @@ test:
test-v:
python3 -m pytest tests/ -v --tb=long
# Lint: deterministic F-rule gate (NameError class); matches the CI quick-test step
lint:
python3 -m ruff check . --select F
# Run codebase health check (requires ouroboros importable)
health:
python3 -c "from ouroboros.review import compute_complexity_metrics; \
python3 -c "from ouroboros.review import collect_sections, compute_complexity_metrics; \
import pathlib, json; \
m = compute_complexity_metrics(pathlib.Path('.')); \
print(json.dumps(m, indent=2, default=str))"
sections, stats = collect_sections(pathlib.Path('.'), pathlib.Path('../data')); \
m = compute_complexity_metrics(sections); \
print(json.dumps({'repo': stats, **m}, indent=2, default=str))"
# Clean Python cache files
clean:

4
README.md
View file

@ -7,7 +7,7 @@
[![Linux](https://img.shields.io/badge/Linux-x86__64-orange.svg)](https://github.com/razzant/ouroboros/releases)
[![Windows](https://img.shields.io/badge/Windows-x64-blue.svg)](https://github.com/razzant/ouroboros/releases)
[![OuroborosHub](https://img.shields.io/badge/OuroborosHub-skills%20marketplace-8A2BE2.svg)](https://github.com/razzant/OuroborosHub)
[![Version 6.25.0-rc.3](https://img.shields.io/badge/version-6.25.0--rc.3-green.svg)](VERSION)
[![Version 6.26.0-rc.1](https://img.shields.io/badge/version-6.26.0--rc.1-green.svg)](VERSION)
A self-modifying AI agent that writes its own code, rewrites its own mind, and evolves autonomously. Born February 16, 2026.
@ -520,6 +520,7 @@ the contribution guide only routes to those sources.
| Version | Date | Description |
|---------|------|-------------|
| 6.26.0-rc.1 | 2026-06-10 | **release: systemic hardening — immune system, custody, memory integrity, native multimodal chat.** Provider registry SSOT + credential-aware lanes; locked RMW state (`update_json_locked`, queue locks, visible supervisor death); memory integrity (atomic+quarantined dialogue blocks, honest scratchpad journal, full Pattern Register window); immune hardening (triad anti-refusal NO_FINDINGS contract, BIBLE P3 loud-advisory bound with durable `advisory_overrides`, dispatcher-level `mutates_worktree` invalidation, fail-closed scope checklist, `find` de-whitelisted); security (file-browser symlink containment, HMAC session cookies, SSRF metadata guard, zip-slip hardening, single-exec dispatch, SHA256-pinned runtime download); process custody (`process_custody.py` ledger + strict-fingerprint reaper + parent lifelines); native multimodal chat (vision capability map, web uploads and browser screenshots as native image blocks, K=3 eviction with captions, image-aware token estimates); deterministic ruff F-gate in CI. |
| 6.25.0-rc.3 | 2026-06-10 | **refactor(consolidation): first function-count paydown and gate headroom.** Owner-approved consolidation pass: remove 14 dead functions, fold ~10 duplicate-body clusters into SSOT survivors, inline ~30 trivial wrappers (3009 → 2953 gated functions, behavior preserved); raise `MAX_TOTAL_FUNCTIONS` to 3500 to stop gate churn after external PR merges landed the branch at zero slack; compress the gate-comment archaeology; drop the dormant file-size-budget parser (its DEVELOPMENT.md section no longer exists); sync DEVELOPMENT.md and add structural-gate rationale to ARCHITECTURE.md. Carries the rc.2 fix-forward surface unchanged. |
| 6.25.0-rc.1 | 2026-06-09 | **feat(code-intelligence): add query_code, ripgrep search, and clearer self-improvement settings.** Adds a deterministic code-intelligence v2 layer with an incremental JSON index, derived-only symbol/reference/call facts for Python and JS/TS, the new read-only `query_code` tool (`symbols`, `definition`, `references`, `callers`, `callees`, `impact`, `structural`, `relevant_files`), and a ripgrep-backed `search_code` path with the existing safety filters and Python fallback. The review atlas and codebase digest consume the upgraded facts without changing review budgets or selection policy. Behavior settings now expose post-task self-improvement through one clear trigger selector, move background-cognition timing into its own section, remove the unused evolution cost-threshold setting, and clarify that Every-N counts every eligible task. Subagent tool calls are mirrored into canonical tool logs for swarm observability. |
| 6.24.0-rc.4 | 2026-06-09 | **fix(self-evolution): fix-forward rc.4 after rc.3 CI exposed a stale browser-isolation assertion.** Keeps the rc.3 server-driven self-evolution hardening intact, updates the browser module-state test to expect both owner-control browser routes (`/api/owner/context-mode` and `/api/settings`), and ships the reviewed fix-forward release metadata after the `v6.24.0-rc.3` tag failed CI. No runtime behavior change beyond the rc.3 hardening. |
@ -529,7 +530,6 @@ the contribution guide only routes to those sources.
| 6.23.3 | 2026-06-08 | **feat(self-evolution): thin per-project facts store + configurable memory carry (C2 / Phase 3).** A project-scoped task (an external/workspace task, or one given an explicit `--project-id`) now keeps its learned facts in a per-project knowledge store under the canonical data dir (`projects/<id>/knowledge`, `ouroboros/project_facts.py`), resolved via S7 (explicit id wins, else a stable hash of the workspace path). The knowledge tool (`tools/knowledge.py`) and context loader (`context.py`) redirect to that store when project-scoped, so prior project facts are available and isolated from the global `memory/knowledge` tree. The store lives under the canonical data dir (not a task's child drive) so it persists across forked/empty runs, and the leak guard (red-team R3.1) suppresses the post-task canonical dual-run for project-scoped tasks so project facts can never contaminate global memory or another project. No per-project identity; non-workspace tasks are unchanged (canonical memory). Configurable memory carry is unchanged in semantics (forked/empty/shared via `prepare_task_drive`); the evolutionary driver uses a persistent isolated data root. |
| 6.23.2 | 2026-06-08 | **feat(self-evolution): improvement-backlog overhaul (C2 / Phase 2).** The durable improvement backlog stops losing signal: recurring items bump a `count` + `last_seen` instead of being dropped (and a recurrence re-opens a closed item); the digest is ranked by `priority` (high/med/low) then recurrence count then recency, so an important older item outranks a junk burst; `close_backlog_items` marks addressed items `done` with `closed_at`, and a promoted item is closed only when its reviewed self-mod commit is restart-verified and absorbed (`apply_pending_request` records `post_task_backlog_id`; `verify_restart` closes it on absorb — not merely on commit); and `groom_backlog` runs an LLM grooming pass (merge near-duplicates, mark resolved, cap the list) on a size-triggered, non-error-gated schedule, re-serialized through the locked parser-safe writer. Reflection nominations now carry optional `priority` and a `kind` (`bug`/`improvement`/`capability_idea`) so forward-looking capability ideas are captured, not only bugs. `CONSCIOUSNESS.md` backlog upkeep now points at the automated grooming instead of hand-editing. Additive schema; existing items remain valid. |
| 6.23.1 | 2026-06-08 | **feat(self-evolution): post-task evolution envelope + budget refusal guard (C1).** Adds an owner-gated, default-OFF post-task self-evolution capability: `OUROBOROS_POST_TASK_EVOLUTION` (+ `_CADENCE` default `llm`, `_BUDGET_USD`) in `config.py`. After a qualifying task the worker may make an LLM-first decision (`ouroboros/post_task_evolution.py`) to promote ONE high-value code-class improvement by writing a durable signal; the supervisor's idle tick applies it through the existing gated `enqueue_evolution_task_if_needed` (idle/restart-verify/3-fail-breaker/reserve/advanced-pro/owner) — never enqueuing from the worker, never from evolution/subagent tasks, preserving the backlog `requires_plan_review` gate, with one-shot autostop. New `supervisor.state.reset_per_task_budget` lets evolutionary drivers reset the per-task budget ONLY in an isolated `OUROBOROS_DATA_DIR` (refuses the live data root, BIBLE P8). A zero-core `devtools/benchmarks/evolve_smoke.py` harness verifies carry + budget-reset + live-untouched on a throwaway clone. Identity stays candidate-only; default behavior unchanged. |
| 6.23.0-rc.1 | 2026-06-08 | **feat(subagents): adaptive progress-aware planning-swarm wait (completes the swarm fan-out/waits set).** `plan_task` planning scouts no longer fail to a fixed cutoff that discards in-flight work: `_collect_planning_handoffs` polls in slices and keeps extending while at least one non-terminal scout is RUNNING with a fresh heartbeat (from `queue_snapshot.json`), up to a generous `OUROBOROS_PLAN_TASK_SWARM_MAX_WAIT_SEC` ceiling (default 900s), then fails closed with a precise `wait_stop_reason` (`stalled`/`saturated`/`ceiling`). The wait/stop decision is a structured fact (terminal status + heartbeat freshness), not keyword matching (P5). Adds a "Timeout & Wait Control" guideline to DEVELOPMENT.md and a pointed progress-aware-waiting line to CHECKLISTS.md item #18. This rc completes the v6.23.0 swarm work begun in 6.22.2 (parallel `schedule_subagent` emission + parent-side lock) and 6.22.3 (fan-out telemetry + accept/reject markers). Dedup is unchanged. |
Older releases are preserved in Git tags and GitHub releases. Older 6.x rows, the 5.2.0 through 5.33.0-rc.6 rows, and former `4.0.0` rows are rolled off to respect the P9 changelog cap; their full bodies remain at their git tags.
---

2
VERSION
View file

@ -1 +1 @@
6.25.0-rc.3
6.26.0-rc.1

2
devtools/benchmarks/swe_bench_pro/grade_pro.py
View file

@ -8,7 +8,7 @@ per-instance output files.
python pro/grade_pro.py --predictions runs/pro_smoke/predictions.jsonl --workers 4
"""
from __future__ import annotations
import argparse, ast, csv, json, os, pathlib, subprocess, sys
import argparse, ast, csv, json, pathlib, subprocess, sys
if __package__ in {None, ""}:
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[3]))

27
docs/ARCHITECTURE.md
View file

@ -1,4 +1,4 @@
# Ouroboros v6.25.0-rc.3 — Architecture & Reference
# Ouroboros v6.26.0-rc.1 — Architecture & Reference
This file is NOT a changelog. Version history lives in README.md, git tags, and commit log.
@ -77,7 +77,7 @@ server.py (Starlette+uvicorn) ← HTTP + WebSocket on configurable host:port (de
├── triad_review.py ← Shared multi-model review primitives: JSON-array extraction is reused by repo + skill review; per-actor records, quorum/degraded accounting, and model-error events power the skill-review path
├── onboarding_wizard.py ← Shared desktop/web onboarding bootstrap + validation
├── settings_setup_contract.py ← SSOT for Settings/Onboarding setup contract, derived bootstrap state, and setup payload validation
├── owner_inject.py ← Per-task user message mailbox (compat module name)
├── owner_mailbox.py ← Per-task user message mailbox (compat module name)
├── launcher_bootstrap.py ← Bundle-to-repo bootstrap and managed sync helpers (used by launcher.py)
├── provider_models.py ← Provider-specific model ID helpers, direct-provider defaults (OpenAI, Anthropic, Cloud.ru, GigaChat)
├── runtime_mode_policy.py ← Runtime-mode protected-path policy (safety-critical files, frozen contracts, release/managed invariants) shared by registry, git tools, and Claude gateway guards
@ -175,6 +175,18 @@ server.py (Starlette+uvicorn) ← HTTP + WebSocket on configurable host:port (de
│ └── subagent_integration.py ← integrate_subagent_patch: parent's manifest-first integration of an acting subagent's workspace.patch. For self_worktree children it applies into ctx.active_repo_dir() (sha256-verified, 3-way --index, protected-path gated, top-only lineage check, genesis refused), stages but never commits. For external_workspace children it verifies the child wrote in the same active external workspace and records an audited verdict without re-applying the patch. Also compare_subagent_patches: read-only best-of-N helper that shows several children's candidate patches side by side for LLM-first synthesis
└── platform_layer.py ← Cross-platform process/path/locking helpers
ouroboros/process_custody.py ← Supervised spawning + durable orphan ledger
(v6.26.0): `spawn_supervised()` records every long-lived child in
`data/state/process_ledger.jsonl` ({pid, pgid, fingerprint{start_time,
cmd_sha256}, purpose, scope task|session|daemon, owner_task, session_id});
the reaper (server startup + 10-min supervisor tick) kills entries whose
generation/task owner is gone, matching by STRICT fingerprint only —
never by command-line class, so dev and packaged instances can coexist.
`start_parent_lifeline()` gives our python entrypoints (workers,
extension runner, claude readonly child) a ppid watchdog that
group-suicides when the parent dies. Panic layers (`_active_subprocesses`,
port sweeps, Windows Job Objects) are unchanged complements.
# Build & CI (not part of runtime)
.github/workflows/ci.yml ← Four-tier CI (quick / full / integration / build+release)
build.sh ← macOS build (PyInstaller → .dmg)
@ -790,7 +802,7 @@ History sync is intentionally two-pass: progress/system entries are replayed fir
### Files
`web/modules/files.js` is the browser file manager: directory tree, breadcrumbs, preview/editor, upload/download, copy/move, and write guards. Backend policy lives in `gateway/files.py`: root confinement is lexical; symlink targets may intentionally resolve outside the root; owner-only state and skill control-plane sidecars are protected.
`web/modules/files.js` is the browser file manager: directory tree, breadcrumbs, preview/editor, upload/download, copy/move, and write guards. Backend policy lives in `gateway/files.py`: root confinement is enforced on the RESOLVED path (v6.26.0) — symlinks whose target leaves the configured root are listed but not readable/writable/traversable; owner-only state and skill control-plane sidecars are protected.
### Skills, Marketplace, Widgets
@ -825,6 +837,13 @@ If `OUROBOROS_NETWORK_PASSWORD` is configured, non-loopback HTTP/WebSocket acces
The executable route SSOT is `ouroboros/gateway/router.py`; file-browser routes come from `gateway/files.py::file_browser_routes()`, the contract index is `gateway/contracts.py::HTTP_ENDPOINTS`, and Host Service routes come from `gateway/host_service.py::create_host_service_app`.
File-browser symlink containment (v6.26.0): every `/api/files/*` endpoint
resolves the requested path and rejects it when the RESOLUTION leaves the
configured root (`Path escapes file browser root`). In-root symlinks keep
working; symlinks pointing outside the root are listed (with
`is_symlink: true`) but cannot be read, written, deleted, or traversed —
the old pass-through behavior was a root escape.
| Method | Path | Handler |
|---|---|---|
| GET | `/` | `server.index_page` |
@ -1333,6 +1352,8 @@ Runtime floors:
| USE_LOCAL_LIGHT | false | Route light model to local server |
| USE_LOCAL_CONSCIOUSNESS | false | Route background consciousness model slot to local server |
| USE_LOCAL_FALLBACK | false | Route fallback model to local server |
| OUROBOROS_MAX_ROUNDS | 200 | Main-loop LLM round ceiling per task (hot-reloadable) |
| OUROBOROS_SKILL_LIFECYCLE_TIMEOUT_SEC | 1800 | Skill lifecycle lane deadline before a wedged job fails loudly |
| OUROBOROS_BG_MAX_ROUNDS | 10 | Max LLM rounds per consciousness cycle |
| OUROBOROS_BG_WAKEUP_MIN | 30 | Min wakeup interval (seconds) |
| OUROBOROS_BG_WAKEUP_MAX | 7200 | Max wakeup interval (seconds) |

7
docs/CHECKLISTS.md
View file

@ -35,6 +35,12 @@ When a new reviewable concern appears, add it here — not in prompts or docs.
`advisory_obligations_acknowledged` to `events.jsonl`; stale advisory still
blocks. Under `blocking`, `commit_reviewed` can proceed only when no open
obligations or commit-readiness debt remain.
- **Loud advisory enforcement (BIBLE P3 bound):** the owner chooses enforcement;
`advisory` is legitimate ONLY while every decision blocking enforcement would
have stopped (critical findings, quorum failure, infrastructure failure,
missing advisory provider) leaves a durable trace: a `review_advisory_override`
event in `events.jsonl` plus the persistent `advisory_overrides_count` /
recent-overrides fields in `review_status`. Silent advisory is forbidden.
- Once advisory is fresh → call commit_reviewed immediately without further edits.
- Bypass (`skip_advisory_review=True`) is an **absolute** escape hatch: it short-circuits the entire commit gate (freshness + open obligations + open commit-readiness debt). Every bypass is durably audited in events.jsonl. Open obligations/debt stay visible in `review_status` (`repo_commit_ready=false`) but do NOT block the bypassed commit. Reach for it when advisory cannot run (provider outage, rate limit) or when the stale signals are known to be obsolete.
@ -112,6 +118,7 @@ or Intent/Scope checklists are.
| 14 | Changing extension loader/dispatch or isolated deps? | Native-risk extension imports and tool/route/WS handlers must stay out-of-process. Add or run regression tests where a native-risk plugin aborts during import and the host survives, plus tool/route child-dispatch tests. Do not "fix" failures by importing native-risk plugin code in `server.py`. |
| 15 | Changing `supervisor/git_ops.py`, `launcher.py`, `server.py`, `ouroboros/tools/review_helpers.py`, `ouroboros/tools/git.py`, tests, or evolution scheduling/checkpoint code? | Prove two invariants before review spend: (1) pytest/preflight cannot mutate the live repo or live `data/` (`OUROBOROS_DATA_DIR` / `OUROBOROS_SETTINGS_PATH` must be isolated, and `OUROBOROS_MANAGED_BY_LAUNCHER` must not leak into test subprocesses); (2) autonomous restart/reset cannot erase active evolution work — it must either land a reviewed local commit or preserve a rescue/transaction recovery pointer and pause/stop the campaign. |
| 16 | Changing `devtools/benchmarks/`? | Confirm it preserves official benchmark boundaries: no replacement scoring, no benchmark-specific prompt/routing hacks, no generated benchmark outputs under `repo/`, no secrets printed or committed, and no runtime-core imports from `devtools/`. Touched `devtools` files are reviewable executable operator code, even though unrelated `devtools` files use Atlas `excluded_dir` coverage-manifest entries and stay compact in broad packs. |
| 17 | Diff spawns OS processes (`subprocess.Popen` / `mp.Process` without a bounded wait)? | Route it through `ouroboros.process_custody.spawn_supervised` (or `record_process` write-through) with an explicit `scope` (`task`/`session`/`daemon`) so the orphan reaper can find it; `tests/test_process_custody.py` enforces the allowlist. |
Rule: read before write. Never reconstruct `VERSION`, `pyproject.toml`
`version`, or the README badge from memory — one stale reconstruction creates

16
docs/DEVELOPMENT.md
View file

@ -589,6 +589,22 @@ Before every commit, verify the following:
---
## Process Custody Rule
Long-lived OS processes (anything `subprocess.Popen`-ed or `mp.Process`-ed
without a bounded wait in the same call) **MUST** be spawned through
`ouroboros.process_custody.spawn_supervised(cmd, drive_root=..., purpose=...,
scope=...)` — or, when an existing manager owns the Popen call, registered via
`record_process(...)` write-through immediately after spawn. The custody
ledger (`data/state/process_ledger.jsonl`) is what lets the orphan reaper find
children after an abrupt worker/server death; an unledgered process orphans
invisibly and forever. Scopes: `task` (dies with its task), `session` (dies
with the server generation), `daemon` (launcher-managed; reaper only prunes).
The reaper kills strictly by (pid, start_time, cmd_sha256) fingerprint — never
add command-line-class matching, which would let a dev instance reap a
packaged instance's processes. `tests/test_process_custody.py` enforces the
chokepoint with an explicit allowlist for bounded synchronous helpers.
## Platform Abstraction Rule
All platform-specific code **MUST** go through `ouroboros/platform_layer.py`.

2
ouroboros/agent.py
View file

@ -38,7 +38,7 @@ from ouroboros.agent_startup_checks import (
verify_system_state,
)
from ouroboros.agent_task_pipeline import (
build_trace_summary, emit_task_results, build_review_context,
emit_task_results, build_review_context,
)
from ouroboros.task_results import STATUS_RUNNING, write_task_result
from ouroboros.contracts.task_constraint import normalize_task_constraint

11
ouroboros/agent_startup_checks.py
View file

@ -6,6 +6,7 @@ import logging
import os
import re
import subprocess
import time
from typing import Any, Dict, Tuple
from ouroboros.utils import atomic_write_json, utc_now_iso, read_text, append_jsonl, read_json_dict
@ -25,8 +26,14 @@ def check_uncommitted_changes(env: Any) -> Tuple[dict, int]:
lock_path = env.repo_path(".git/index.lock")
if lock_path.exists():
try:
lock_path.unlink()
log.warning("Removed stale git index.lock")
# Age gate (mirrors supervisor.git_ops._stale_git_lock_paths):
# an unconditional unlink could yank index.lock from under a
# LIVE supervisor git operation during worker boot, corrupting
# the index. Only locks orphaned by dead processes are stale.
age_sec = time.time() - lock_path.stat().st_mtime
if age_sec >= 15.0:
lock_path.unlink()
log.warning("Removed stale git index.lock (age %.0fs)", age_sec)
except OSError:
pass

55
ouroboros/agent_task_pipeline.py
View file

@ -4,7 +4,6 @@ from __future__ import annotations
import json
import logging
import os
import pathlib
import threading
import time
@ -33,58 +32,8 @@ from ouroboros.utils import utc_now_iso, append_jsonl, truncate_review_artifact
log = logging.getLogger(__name__)
def _resolve_task_summary_model(default_model: str) -> str:
prefix_to_provider = {
"openai::": "openai",
"anthropic::": "anthropic",
"cloudru::": "cloudru",
"gigachat::": "gigachat",
"openai-compatible::": "openai-compatible",
"openrouter::": "openrouter",
}
provider_env_keys: dict[str, list[str]] = {
"openai": ["OPENAI_API_KEY"],
"anthropic": ["ANTHROPIC_API_KEY"],
"cloudru": ["CLOUDRU_FOUNDATION_MODELS_API_KEY"],
"gigachat": ["GIGACHAT_CREDENTIALS"],
"openrouter": ["OPENROUTER_API_KEY"],
}
def model_has_credentials(model: str) -> bool:
name = str(model or "").strip()
provider = "openrouter"
for prefix, candidate_provider in prefix_to_provider.items():
if name.startswith(prefix):
provider = candidate_provider
break
if provider == "openai-compatible":
compat = str(os.environ.get("OPENAI_COMPATIBLE_API_KEY", "") or "").strip()
legacy_key = str(os.environ.get("OPENAI_API_KEY", "") or "").strip()
legacy_base = str(os.environ.get("OPENAI_BASE_URL", "") or "").strip()
return bool(compat or (legacy_key and legacy_base))
if provider == "gigachat":
creds = str(os.environ.get("GIGACHAT_CREDENTIALS", "") or "").strip()
user = str(os.environ.get("GIGACHAT_USER", "") or "").strip()
password = str(os.environ.get("GIGACHAT_PASSWORD", "") or "").strip()
return bool(creds or (user and password))
for env_key in provider_env_keys.get(provider, ["OPENROUTER_API_KEY"]):
if str(os.environ.get(env_key, "") or "").strip():
return True
return False
if model_has_credentials(default_model):
return default_model
for env_name in (
"OUROBOROS_MODEL_LIGHT",
"OUROBOROS_MODEL_FALLBACK",
"OUROBOROS_MODEL",
"OUROBOROS_MODEL_CODE",
):
candidate = str(os.environ.get(env_name, "") or "").strip()
if candidate and model_has_credentials(candidate):
return candidate
return default_model
# Credential-aware model selection lives in the provider registry SSOT.
from ouroboros.provider_models import resolve_credentialed_model as _resolve_task_summary_model
def build_trace_summary(llm_trace: dict) -> str:

3
ouroboros/cli.py
View file

@ -802,7 +802,8 @@ def _patch_from_result(
def _is_terminal_result(result: Dict[str, Any]) -> bool:
status = str(result.get("status") or "").lower()
if status not in {"completed", "failed", "cancelled", "rejected_duplicate"}:
from ouroboros.task_status import SETTLED_STATUSES
if status not in SETTLED_STATUSES:
return False
bundle = result.get("artifact_bundle") if isinstance(result.get("artifact_bundle"), dict) else {}
artifact_status = str(bundle.get("status") or result.get("artifact_status") or "").lower()

7
ouroboros/config.py
View file

@ -13,7 +13,7 @@ import pathlib
import re
import sys
import time
from typing import Optional
from typing import Any, Optional
from ouroboros.platform_layer import pid_lock_acquire as _compat_pid_lock_acquire
from ouroboros.platform_layer import pid_lock_release as _compat_pid_lock_release
@ -103,6 +103,10 @@ SETTINGS_DEFAULTS = {
"OUROBOROS_PLAN_TASK_SWARM_HEARTBEAT_STALE_SEC": 120,
"TOTAL_BUDGET": 10.0,
"OUROBOROS_PER_TASK_COST_USD": 20.0,
# Main-loop round ceiling (was an inline literal in loop.py — hot-reloadable now).
"OUROBOROS_MAX_ROUNDS": 200,
# Skill lifecycle lane deadline (wedged-job loud-failure bound).
"OUROBOROS_SKILL_LIFECYCLE_TIMEOUT_SEC": 1800,
"OUROBOROS_SOFT_TIMEOUT_SEC": 600,
"OUROBOROS_HARD_TIMEOUT_SEC": 1800,
"OUROBOROS_FINALIZATION_GRACE_SEC": FINALIZATION_GRACE_DEFAULT_SEC,
@ -991,6 +995,7 @@ def apply_settings_to_env(settings: dict) -> None:
"OUROBOROS_PLAN_TASK_SWARM_HEARTBEAT_STALE_SEC",
"TOTAL_BUDGET", "OUROBOROS_PER_TASK_COST_USD", "GITHUB_TOKEN", "GITHUB_REPO",
"OUROBOROS_TOOL_TIMEOUT_SEC", "OUROBOROS_FINALIZATION_GRACE_SEC",
"OUROBOROS_MAX_ROUNDS",
"OUROBOROS_BG_MAX_ROUNDS", "OUROBOROS_BG_WAKEUP_MIN", "OUROBOROS_BG_WAKEUP_MAX",
"OUROBOROS_WEBSEARCH_MODEL",
"OUROBOROS_POST_TASK_EVOLUTION", "OUROBOROS_POST_TASK_EVOLUTION_CADENCE",

8
ouroboros/consciousness.py
View file

@ -9,9 +9,11 @@ import os
import pathlib
import queue
import threading
import time
import traceback
from typing import Any, Callable, Dict, List, Optional
from typing import TYPE_CHECKING, Any, Callable, Dict, List, Optional
if TYPE_CHECKING:
from ouroboros.tools.registry import ToolRegistry
from ouroboros.loop_tool_execution import StatefulToolExecutor, _truncate_tool_result
from ouroboros.utils import (
@ -530,7 +532,7 @@ class BackgroundConsciousness:
def _build_registry(self) -> "ToolRegistry":
"""Create a ToolRegistry scoped to background-allowed tools."""
from ouroboros.tools.registry import ToolRegistry, ToolContext, ToolEntry
from ouroboros.tools.registry import ToolRegistry, ToolEntry
registry = ToolRegistry(repo_dir=self._repo_dir, drive_root=self._drive_root)

104
ouroboros/consolidator.py
View file

@ -2,7 +2,6 @@ import json
import logging
import os
import pathlib
import hashlib
from typing import Any, Dict, List, Optional, Tuple
from ouroboros.contracts.chat_id_policy import is_a2a_chat_id
@ -20,6 +19,19 @@ BLOCK_SIZE = 100 # Messages per consolidation block
MAX_SUMMARY_BLOCKS = 10 # Compress into era when exceeded
ERA_COMPRESS_COUNT = 4 # Oldest blocks to compress per era
CONSOLIDATION_MODEL = "google/gemini-3.5-flash"
def _consolidation_model() -> str:
"""Credential-aware consolidation model (Provider Independence invariant).
The default routes through OpenRouter; on single-direct-provider installs
that would silently fail every consolidation pass and freeze long-term
memory (offset never advances). Resolve through the provider registry so
any credentialed configured lane keeps memory consolidation alive.
"""
from ouroboros.provider_models import resolve_credentialed_model
return resolve_credentialed_model(CONSOLIDATION_MODEL)
CONSOLIDATION_REASONING_EFFORT = "medium"
def should_consolidate(
@ -169,7 +181,7 @@ def _call_consolidation_llm(llm_client: Any, prompt: str, label: str) -> Tuple[s
try:
msg, usage = llm_client.chat(
messages=[{"role": "user", "content": prompt}],
model=CONSOLIDATION_MODEL,
model=_consolidation_model(),
tools=None,
reasoning_effort="low",
max_tokens=16384,
@ -283,19 +295,58 @@ def _load_blocks(path: pathlib.Path) -> List[Dict[str, Any]]:
try:
return json.loads(read_text(path))
except (json.JSONDecodeError, ValueError):
log.warning("Corrupt blocks file %s, starting fresh", path)
# Memory loss must never be silent (P1): quarantine the corrupt store
# for forensic recovery instead of overwriting it on the next write.
quarantine = path.with_name(f"{path.name}.corrupt-{utc_now_iso().replace(':', '')}.bak")
try:
os.replace(path, quarantine)
log.error("Corrupt blocks file %s — quarantined to %s, starting fresh", path, quarantine)
except OSError:
log.error("Corrupt blocks file %s — quarantine failed, starting fresh", path, exc_info=True)
try:
from ouroboros.utils import append_jsonl
append_jsonl(path.parent.parent / "logs" / "events.jsonl", {
"ts": utc_now_iso(),
"type": "memory_store_corrupt",
"path": str(path),
"quarantine": str(quarantine),
})
except Exception:
log.debug("Failed to emit memory_store_corrupt event", exc_info=True)
return []
def _write_locked_json(path: pathlib.Path, payload: Any) -> None:
"""Write JSON under the cross-process write lock, atomically.
The lock serializes concurrent consolidators; the write itself goes
through a temp file + rename so a crash mid-write can never leave a
truncated dialogue_blocks.json (the long-term memory store).
"""
_mutate_locked_json_list(path, lambda _current: payload)
def _mutate_locked_json_list(path: pathlib.Path, mutator: Any) -> Any:
"""Locked read-modify-write for a JSON list store (atomic replace).
``mutator(current_list) -> new_list`` runs while the sidecar lock is held,
so concurrent appenders cannot be lost between the re-read and the write.
"""
path.parent.mkdir(parents=True, exist_ok=True)
fd = None
try:
fd = os.open(str(path), os.O_RDWR | os.O_CREAT, 0o644)
fd = os.open(str(path) + ".lock", os.O_RDWR | os.O_CREAT, 0o644)
_lock_ex(fd)
os.lseek(fd, 0, os.SEEK_SET)
os.ftruncate(fd, 0)
os.write(fd, json.dumps(payload, ensure_ascii=False, indent=2).encode("utf-8"))
current: Any = []
if path.exists():
try:
current = json.loads(read_text(path))
except (json.JSONDecodeError, ValueError):
current = []
updated = mutator(current if isinstance(current, list) else [])
atomic_write_json(path, updated)
return updated
finally:
if fd is not None:
try:
@ -308,19 +359,7 @@ def _load_meta(path: pathlib.Path) -> Dict[str, Any]:
return read_json_dict(path) or {}
def _chat_log_signature(path: pathlib.Path) -> Dict[str, Any]:
if not path.exists():
return {}
try:
stat = path.stat()
with path.open("r", encoding="utf-8") as handle:
first = next((line.strip() for line in handle if line.strip()), "")
return {
"first_line_sha256": hashlib.sha256(first.encode("utf-8", errors="replace")).hexdigest(),
"size": int(stat.st_size),
}
except OSError:
return {}
from ouroboros.utils import jsonl_generation_signature as _chat_log_signature
def _count_lines(path: pathlib.Path) -> int:
@ -368,7 +407,9 @@ def _rebuild_knowledge_index(knowledge_dir: pathlib.Path) -> None:
except Exception:
log.warning("Failed to rebuild knowledge index", exc_info=True)
SCRATCHPAD_CONSOLIDATION_THRESHOLD = 30000
from ouroboros.context_budget import (
SCRATCHPAD_CONSOLIDATION_THRESHOLD_CHARS as SCRATCHPAD_CONSOLIDATION_THRESHOLD,
)
def should_consolidate_scratchpad(memory: Any) -> bool:
@ -405,7 +446,6 @@ def _consolidate_scratchpad_blocks(
compress_count = max(2, len(blocks) // 2)
old_blocks = blocks[:compress_count]
recent_blocks = blocks[compress_count:]
old_content = "\n\n---\n\n".join(
f"[{b.get('ts', '?')[:16]} \u2014 {b.get('source', '?')}]\n{b.get('content', '')}"
@ -439,7 +479,7 @@ Respond with JSON only (no fences):
try:
msg, usage = llm_client.chat(
messages=[{"role": "user", "content": prompt}],
model=CONSOLIDATION_MODEL,
model=_consolidation_model(),
reasoning_effort="low",
max_tokens=16384,
use_local=os.environ.get("USE_LOCAL_LIGHT", "").lower() in ("true", "1"),
@ -463,9 +503,23 @@ Respond with JSON only (no fences):
"source": "consolidation",
"content": compressed_text.strip(),
}
new_blocks = [compressed_block] + recent_blocks
_write_locked_json(memory.scratchpad_blocks_path(), new_blocks)
# Merge-aware replace UNDER the write lock: blocks appended DURING the
# slow LLM call live only on disk — building the new list from the
# pre-call snapshot would silently drop them. Re-read inside the lock
# and keep every block outside the compressed window (ts+source key).
compressed_keys = {
(str(b.get("ts") or ""), str(b.get("source") or "")) for b in old_blocks
}
def _merge_survivors(live_blocks: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
survivors = [
b for b in live_blocks
if (str(b.get("ts") or ""), str(b.get("source") or "")) not in compressed_keys
]
return [compressed_block] + survivors
new_blocks = _mutate_locked_json_list(memory.scratchpad_blocks_path(), _merge_survivors)
memory.regenerate_scratchpad_md()
log.info("Scratchpad blocks consolidated: %d blocks (%d chars) -> %d blocks (%d chars)",

23
ouroboros/context.py
View file

@ -18,6 +18,8 @@ from ouroboros.context_budget import (
CONTEXT_SOFT_CAP_TOKENS,
LARGE_CONTEXT_SECTION_CHARS,
MAX_RECENT_CHAT_TAIL,
SCRATCHPAD_BLOAT_WARN_CHARS,
SCRATCHPAD_SECTION_BUDGET_CHARS,
)
from ouroboros.context_layout import (
architecture_context_section,
@ -65,7 +67,13 @@ def build_user_content(task: Dict[str, Any]) -> Any:
combined_text = "\n".join(part for part in (image_caption, text if text != image_caption else "") if part) or "Analyze the screenshot"
return [
{"type": "text", "text": combined_text},
{"type": "image_url", "image_url": {"url": f"data:{task.get('image_mime', 'image/jpeg')};base64,{image_b64}"}},
{
"type": "image_url",
"image_url": {"url": f"data:{task.get('image_mime', 'image/jpeg')};base64,{image_b64}"},
# Eviction metadata (stripped before provider calls): the K-newest
# image policy replaces older blocks with this caption.
"_caption": str(image_caption or "")[:200],
},
]
@ -271,7 +279,7 @@ def build_governance_sections(env: Any, *, warn_large: bool = False, warn_label:
return sections
_SECTION_BUDGETS = {"scratchpad": 90_000, "identity": 80_000, "registry": 30_000}
_SECTION_BUDGETS = {"scratchpad": SCRATCHPAD_SECTION_BUDGET_CHARS, "identity": 80_000, "registry": 30_000}
def _warn_if_over_budget(name: str, content: str) -> None:
@ -395,7 +403,9 @@ def build_recent_sections(memory: Memory, env: Any, task_id: str = "") -> List[s
consolidated_offset,
_chat_tail,
)
chat_summary = memory.summarize_chat(chat_entries)
# Pass the same tail intent down: summarize_chat's internal default cap
# would silently re-cut the low-mode full-window read to 1000 lines.
chat_summary = memory.summarize_chat(chat_entries, limit=_chat_tail)
if chat_summary:
sections.append("## Recent chat\n\n" + chat_summary)
@ -646,7 +656,7 @@ def build_health_invariants(env: Any) -> str:
sp_len = len(read_text(env.drive_path("memory/scratchpad.md")).strip())
if sp_len < 50:
checks.append("WARNING: EMPTY SCRATCHPAD — scratchpad is nearly empty. Memory loss signal.")
elif sp_len > 50000:
elif sp_len > SCRATCHPAD_BLOAT_WARN_CHARS:
checks.append(f"WARNING: BLOATED SCRATCHPAD — {sp_len} chars. Extract durable insights to knowledge base.")
else:
checks.append(f"OK: scratchpad size ({sp_len} chars)")
@ -969,11 +979,6 @@ def apply_message_token_soft_cap(
return messages, info
from ouroboros.context_compaction import (
_COMPACTION_PROTECTED_TOOLS,
compact_tool_history,
compact_tool_history_llm,
)
def safe_read(path: pathlib.Path, fallback: str = "") -> str:

17
ouroboros/context_budget.py
View file

@ -53,3 +53,20 @@ MAX_RECENT_CHAT_TAIL = 1000
# Low fires emergency tool-history compaction earlier (~100K tokens at chars/4)
# to fit a ~200K window, and (unlike max) also enables remote routine compaction.
LOW_EMERGENCY_COMPACTION_CHARS = 400_000
# --- Native image blocks (v6.26.0 multimodal chat) ---------------------------
# Char-equivalent for ONE image block in chars/4 token estimates (~1.1K tokens):
# vision models bill per tile, not per base64 char.
IMAGE_BLOCK_CHAR_EQUIVALENT = 4_400
# Live image blocks kept in the transcript (single counter across owner
# uploads, browser screenshots, and transport injections). Older images are
# replaced by a caption placeholder pointing to the re-view path.
MAX_LIVE_IMAGE_BLOCKS = 3
# --- Scratchpad size thresholds (SSOT; previously scattered literals) -------
# Context-section soft budget for the rendered scratchpad (warn-only).
SCRATCHPAD_SECTION_BUDGET_CHARS = 90_000
# Health-invariant bloat warning ("extract durable insights to knowledge").
SCRATCHPAD_BLOAT_WARN_CHARS = 50_000
# Block-storage consolidation trigger (consolidator compresses oldest blocks).
SCRATCHPAD_CONSOLIDATION_THRESHOLD_CHARS = 30_000

19
ouroboros/context_compaction.py
View file

@ -147,7 +147,22 @@ def _render_round_block(messages: list, start: int, end: int) -> str:
lines.append(f"TOOL_RESULT {tool_name}:")
lines.append(_excerpt_for_summary(content))
elif role == "user":
content = str(msg.get("content") or "")
content = msg.get("content")
if isinstance(content, list):
# Multipart user content: never str() an image block (the
# base64 payload would flood the light summarizer's prompt).
parts = []
for block in content:
if not isinstance(block, dict):
parts.append(str(block))
elif str(block.get("type") or "") in ("image_url", "image"):
caption = str(block.get("_caption") or "").strip()
parts.append(f"[image: {caption or 'omitted'}]")
else:
parts.append(str(block.get("text", "")))
content = "\n".join(part for part in parts if part)
else:
content = str(content or "")
lines.append("USER_INPUT:")
lines.append(_excerpt_for_summary(content))
return "\n".join(lines).strip()
@ -254,7 +269,7 @@ def compact_tool_history_llm(
return messages, None
spans_to_keep = spans[-keep_recent:]
keep_starts = {start for start, _ in spans_to_keep}
{start for start, _ in spans_to_keep}
compactable_spans = []
protected_starts = set()
for start, end in spans[:-keep_recent]:

2
ouroboros/contracts/plugin_api.py
View file

@ -10,7 +10,7 @@ from __future__ import annotations
import enum
import pathlib
from typing import Any, Awaitable, Callable, Dict, List, Protocol, Sequence, runtime_checkable
from typing import Any, Awaitable, Callable, Dict, Protocol, Sequence, runtime_checkable
from ouroboros.skill_token import SkillToken

64
ouroboros/deep_self_review.py
View file

@ -15,12 +15,12 @@ from ouroboros.tools.review_context_atlas import ( # noqa: E402
compile_review_context_atlas,
)
from ouroboros.tools.review_helpers import ( # noqa: E402
_BINARY_SNIFF_BYTES,
_MAX_FULL_REPO_FILE_BYTES,
_is_probably_binary,
REVIEW_PROMPT_TOKEN_BUDGET,
)
from ouroboros.utils import atomic_write_json, estimate_tokens, utc_now_iso # noqa: E402
from ouroboros.config import get_context_mode, get_deep_self_review_model, resolve_effort # noqa: E402
from ouroboros.provider_models import provider_for_model, provider_has_credentials # noqa: E402
from ouroboros.context_layout import generate_doc_nav_map # noqa: E402
# Non-agent visual assets.
@ -28,6 +28,18 @@ _SKIP_DIR_PREFIXES = (
"assets/",
)
# Output reservation inside the reviewer's 1M window (same class of fix as
# scope_review._SCOPE_INPUT_TOKEN_LIMIT): 920K input + 100K output exceeds 1M
# and yields a deterministic provider 400, so the assembled INPUT prompt is
# gated on min(SSOT budget, window − output − tokenizer margin).
_DEEP_MAX_OUTPUT_TOKENS = 100_000
_DEEP_MODEL_CONTEXT_WINDOW = 1_000_000
_DEEP_OUTPUT_MARGIN_TOKENS = 155_000
_DEEP_INPUT_TOKEN_LIMIT = min(
REVIEW_PROMPT_TOKEN_BUDGET,
_DEEP_MODEL_CONTEXT_WINDOW - _DEEP_MAX_OUTPUT_TOKENS - _DEEP_OUTPUT_MARGIN_TOKENS,
)
_MEMORY_WHITELIST = [
"memory/identity.md",
"memory/scratchpad.md",
@ -162,8 +174,8 @@ def build_review_pack(
tracked_paths=tuple(tracked),
already_included=already_included,
fixed_prompt_tokens=atlas_fixed_tokens,
target_total_tokens=850_000,
hard_total_tokens=920_000,
target_total_tokens=min(850_000, _DEEP_INPUT_TOKEN_LIMIT),
hard_total_tokens=_DEEP_INPUT_TOKEN_LIMIT,
include_tests=False,
title="Generated Deep Self-Review Atlas",
)
@ -197,29 +209,27 @@ def build_review_pack(
def is_review_available() -> Tuple[bool, Optional[str]]:
"""Return whether a suitable large-context review model is configured."""
"""Return whether a suitable large-context review model is configured.
Provider/credential knowledge comes from the provider registry SSOT; the
one deliberate deep-review-specific rule kept here: ``openai::`` is only
trusted when ``OPENAI_BASE_URL`` is unset (a redirected endpoint cannot be
assumed to honor the 1M-context contract this review depends on).
"""
configured = get_deep_self_review_model()
if configured.startswith("openai::"):
if os.environ.get("OPENAI_API_KEY") and not os.environ.get("OPENAI_BASE_URL"):
provider = provider_for_model(configured)
if provider == "openai":
if provider_has_credentials("openai") and not os.environ.get("OPENAI_BASE_URL"):
return True, configured
return False, None
if configured.startswith("openai/"):
if os.environ.get("OPENROUTER_API_KEY"):
# OpenRouter route with a direct-OpenAI rewrite fallback.
if provider_has_credentials("openrouter"):
return True, configured
if os.environ.get("OPENAI_API_KEY") and not os.environ.get("OPENAI_BASE_URL"):
if provider_has_credentials("openai") and not os.environ.get("OPENAI_BASE_URL"):
return True, "openai::" + configured.split("/", 1)[1]
return False, None
if configured.startswith("anthropic::"):
return (True, configured) if os.environ.get("ANTHROPIC_API_KEY") else (False, None)
if configured.startswith("cloudru::"):
return (True, configured) if os.environ.get("CLOUDRU_FOUNDATION_MODELS_API_KEY") else (False, None)
if configured.startswith("gigachat::"):
has_giga = bool(os.environ.get("GIGACHAT_CREDENTIALS") or (os.environ.get("GIGACHAT_USER") and os.environ.get("GIGACHAT_PASSWORD")))
return (True, configured) if has_giga else (False, None)
if configured.startswith("openai-compatible::"):
has_compat = bool(os.environ.get("OPENAI_COMPATIBLE_API_KEY") or (os.environ.get("OPENAI_API_KEY") and os.environ.get("OPENAI_BASE_URL")))
return (True, configured) if has_compat else (False, None)
if os.environ.get("OPENROUTER_API_KEY"):
if provider_has_credentials(provider):
return True, configured
return False, None
@ -253,16 +263,18 @@ def run_deep_self_review(
+ (f", {len(stats['skipped'])} skipped" if stats["skipped"] else "")
)
# Gate full system+pack like scope/plan review; chars/4 undercounts near
# the 1M window, so the prompt budget remains a best-effort guard.
from ouroboros.tools.review_helpers import REVIEW_PROMPT_TOKEN_BUDGET
# Gate full system+pack like scope review: reserve output headroom
# inside the 1M window (min(SSOT, window − output − margin)) so a large
# pack cannot trigger the deterministic input+output>window provider 400.
full_prompt_chars = len(_SYSTEM_PROMPT) + len(pack_text)
estimated_tokens = estimate_tokens(_SYSTEM_PROMPT + pack_text)
if estimated_tokens > REVIEW_PROMPT_TOKEN_BUDGET:
if estimated_tokens > _DEEP_INPUT_TOKEN_LIMIT:
return (
f"❌ Review pack too large: ~{estimated_tokens:,} tokens "
f"({full_prompt_chars:,} chars of system+pack, {stats['file_count']} files). "
f"Maximum is ~{REVIEW_PROMPT_TOKEN_BUDGET:,} tokens. Reduce codebase size or split review."
f"Maximum is ~{_DEEP_INPUT_TOKEN_LIMIT:,} tokens "
f"(window minus {_DEEP_MAX_OUTPUT_TOKENS:,} output reserve). "
"Reduce codebase size or split review."
), {}
if not model:
@ -306,7 +318,7 @@ def run_deep_self_review(
model=model,
tools=None,
reasoning_effort=resolve_effort("deep_self_review"),
max_tokens=100_000,
max_tokens=_DEEP_MAX_OUTPUT_TOKENS,
temperature=None,
no_proxy=True,
)

17
ouroboros/extension_companion.py
View file

@ -131,6 +131,23 @@ class CompanionSupervisor:
popen_kwargs.update(subprocess_new_group_kwargs())
popen_kwargs = merge_hidden_kwargs(popen_kwargs)
proc = subprocess.Popen(descriptor.command, **popen_kwargs) # noqa: S603
# Write-through into the custody ledger (daemon scope: companions
# are launcher-managed and must survive server generations; the
# reaper only prunes their dead entries). The launcher-facing
# extension_companions.json contract stays untouched.
try:
from ouroboros.config import DATA_DIR as _data_dir
from ouroboros.process_custody import record_process
record_process(
pathlib.Path(_data_dir),
pid=proc.pid,
cmd=list(descriptor.command),
purpose=f"companion:{descriptor.skill_name}:{descriptor.name}",
scope="daemon",
)
except Exception:
log.debug("companion custody record failed", exc_info=True)
job_handle = None
if IS_WINDOWS and os.environ.get("OUROBOROS_MANAGED_BY_LAUNCHER") != "1":
job_handle = create_kill_on_close_job()

40
ouroboros/extension_process_runner.py
View file

@ -594,13 +594,39 @@ async def _call_tool(surface: str, args: Dict[str, Any], drive_root: pathlib.Pat
ToolContext(repo_dir=repo_dir, drive_root=drive_root),
dict(ctx_payload or {}),
)
try:
result = handler(ctx, **dict(args or {}))
except TypeError:
result = handler(**dict(args or {}))
# Signature-based dispatch: the old try/except TypeError retry re-EXECUTED
# a handler whose body raised TypeError after side effects had happened.
result = (
handler(ctx, **dict(args or {}))
if _handler_wants_ctx(handler)
else handler(**dict(args or {}))
)
return await _run_maybe_async(result)
def _handler_wants_ctx(handler: Any) -> bool:
"""True when the handler's first parameter is a ctx slot (canonical form).
Extension tool handlers are either ``fn(ctx, **args)`` (canonical) or
``fn(**args)`` / ``fn(named=...)`` (ctx-less). Dispatch on the first
parameter's name so the args dict can still bind named parameters.
"""
import inspect
try:
params = list(inspect.signature(handler).parameters.values())
except (TypeError, ValueError):
return True # builtins/C callables: keep the historical ctx-first call
if not params:
return False
first = params[0]
if first.kind == first.VAR_POSITIONAL:
return True
if first.kind in (first.POSITIONAL_ONLY, first.POSITIONAL_OR_KEYWORD):
return first.name in {"ctx", "context", "_ctx", "tool_context"}
return False
async def _request_from_payload(payload: Dict[str, Any], drive_root: pathlib.Path, repo_dir: pathlib.Path) -> Request:
body = base64.b64decode(str(payload.get("body_b64") or ""))
sent = False
@ -775,4 +801,10 @@ def _child_main(input_path: str) -> None:
if __name__ == "__main__":
if len(sys.argv) != 2:
raise SystemExit("usage: python -m ouroboros.extension_process_runner <payload.json>")
try:
from ouroboros.process_custody import start_parent_lifeline
start_parent_lifeline(label="extension-runner")
except Exception:
pass
_child_main(sys.argv[1])

12
ouroboros/gateway/contracts.py
View file

@ -14,6 +14,17 @@ except ImportError: # pragma: no cover - CI supports Python 3.10.
from typing_extensions import Literal, NotRequired, Required, TypedDict # type: ignore[assignment]
class ChatAttachmentInbound(TypedDict, total=False):
"""One uploaded chat attachment reference (file already stored by
/api/chat/upload under data/uploads/; ``filename`` is the stored
basename). Image attachments are delivered to vision models as native
image blocks (v6.26.0)."""
filename: str
display_name: str
mime: str
class ChatInbound(TypedDict):
"""Inbound WS chat message. ``type`` and ``content`` are required."""
@ -22,6 +33,7 @@ class ChatInbound(TypedDict):
sender_session_id: NotRequired[str]
client_message_id: NotRequired[str]
force_plan: NotRequired[bool]
attachments: NotRequired[list] # list[ChatAttachmentInbound] (additive, v6.26.0)
class TaskConstraintInbound(TypedDict, total=False):

32
ouroboros/gateway/control.py
View file

@ -3,7 +3,7 @@
from __future__ import annotations
import asyncio
import pathlib
import logging
import time
from typing import Any, Dict
@ -16,6 +16,8 @@ from ouroboros.gateway.ws import broadcast_ws_sync
from ouroboros.outcomes import public_task_result
from ouroboros.utils import utc_now_iso
log = logging.getLogger(__name__)
_RECENT_VISIBLE_COMMANDS: Dict[str, float] = {}
_VISIBLE_COMMAND_DEDUPE_SEC = 5.0
_evo_cache: Dict[str, Any] = {}
@ -204,6 +206,16 @@ async def api_update_check(_request: Request) -> JSONResponse:
return json_exception(exc)
def _respawn_workers_after_failed_update() -> None:
"""Revive workers when an update aborts after they were stopped (no restart follows)."""
try:
from supervisor.workers import spawn_workers
spawn_workers()
except Exception:
log.warning("update_apply: failed to respawn workers after aborted update", exc_info=True)
async def api_update_apply(request: Request) -> JSONResponse:
"""Prepare a managed update and restart so safe_restart applies it."""
body = await request_json_or(request, {}, exceptions=(Exception,))
@ -214,20 +226,36 @@ async def api_update_apply(request: Request) -> JSONResponse:
ok, payload = prepare_managed_update(strategy)
if not ok:
return JSONResponse(payload, status_code=409)
# Stop workers AFTER the update is validated/prepared but BEFORE the
# hard reset of the live repo: a self-modifying task writing between
# the rescue snapshot and the reset would have its edits destroyed
# unrescued. Doing this pre-validation killed all workers on a plain
# 409 (no update available) with no restart to revive them.
try:
from supervisor.workers import kill_workers
kill_workers(
result_reason="Task interrupted by an owner-requested managed update (restart follows).",
terminal_status="interrupted",
)
except Exception:
log.warning("update_apply: failed to stop workers before reset", exc_info=True)
try:
checkout_ok, checkout_msg = checkout_and_reset(
BRANCH_DEV,
reason="ui_update_apply",
unsynced_policy="ignore",
unsynced_policy="rescue_and_reset",
)
except Exception as checkout_exc:
_clear_update_intent()
_respawn_workers_after_failed_update()
return JSONResponse(
{"error": f"Prepared update but checkout failed: {checkout_exc}", **payload},
status_code=409,
)
if not checkout_ok:
_clear_update_intent()
_respawn_workers_after_failed_update()
return JSONResponse(
{"error": f"Prepared update but checkout failed: {checkout_msg}", **payload},
status_code=409,

23
ouroboros/gateway/files.py
View file

@ -98,6 +98,15 @@ def _resolve_target(request: Request, rel_path: str) -> tuple[pathlib.Path, path
except ValueError as exc:
raise ValueError("Path escapes file browser root.") from exc
resolved = requested.resolve(strict=False)
# Symlink containment: the lexical check above cannot see where a link
# POINTS. Every endpoint operates within the configured root, so a path
# whose resolution leaves the root (e.g. root/link -> /etc) is rejected
# outright — including symlinks themselves (deleting such a link via the
# API is intentionally blocked rather than special-cased).
try:
resolved.relative_to(root_dir)
except ValueError as exc:
raise ValueError("Path escapes file browser root (symlink target outside root).") from exc
return root_dir, requested, resolved
@ -348,7 +357,7 @@ async def api_files_read(request: Request) -> JSONResponse:
try:
if not rel_path:
return json_error("Missing path.", status=400)
root_dir, target, _ = _resolve_target(request, rel_path)
root_dir, _requested, target = _resolve_target(request, rel_path)
if not target.exists():
return JSONResponse({"error": f"Path not found: {rel_path}"}, status_code=404)
if not target.is_file():
@ -403,7 +412,7 @@ async def api_files_download(request: Request) -> FileResponse | JSONResponse:
try:
if not rel_path:
return json_error("Missing path.", status=400)
_, target, _ = _resolve_target(request, rel_path)
_, _requested, target = _resolve_target(request, rel_path)
if not target.exists():
return JSONResponse({"error": f"Path not found: {rel_path}"}, status_code=404)
if not target.is_file():
@ -422,7 +431,7 @@ async def api_files_content(request: Request) -> FileResponse | JSONResponse:
try:
if not rel_path:
return json_error("Missing path.", status=400)
_, target, _ = _resolve_target(request, rel_path)
_, _requested, target = _resolve_target(request, rel_path)
if not target.exists():
return JSONResponse({"error": f"Path not found: {rel_path}"}, status_code=404)
if not target.is_file():
@ -762,16 +771,8 @@ def file_browser_routes() -> list[Route]:
]
import mimetypes
import os
import pathlib
import uuid
from starlette.datastructures import UploadFile
from starlette.requests import Request
from starlette.responses import JSONResponse
_CHAT_UPLOAD_MAX_BYTES = 50 * 1024 * 1024 # 50 MB
_CHUNK = 64 * 1024 # 64 KB

20
ouroboros/gateway/settings.py
View file

@ -129,6 +129,20 @@ def _build_network_meta(bind_host: str, bind_port: int) -> dict:
}
# Password-class secrets are usually short human-chosen strings: an 8-char
# prefix can BE most of the password. They mask to a constant placeholder;
# long machine-generated API keys keep the recognizable 8-char prefix.
_PASSWORD_CLASS_KEYS = {
"OUROBOROS_NETWORK_PASSWORD",
"GIGACHAT_PASSWORD",
"GIGACHAT_CREDENTIALS",
}
def _mask_password_class(value: Any) -> str:
return "***set***" if str(value or "").strip() else ""
def _mask_secret_value(value: Any) -> str:
text = str(value or "")
return text[:8] + "..." if len(text) > 8 else "***"
@ -501,7 +515,11 @@ async def api_settings_get(request: Request) -> JSONResponse:
safe = {k: v for k, v in settings.items()}
for key in _SECRET_SETTING_KEYS:
if safe.get(key):
safe[key] = _mask_secret_value(safe[key])
safe[key] = (
_mask_password_class(safe[key])
if key in _PASSWORD_CLASS_KEYS
else _mask_secret_value(safe[key])
)
safe["MCP_SERVERS"] = _mask_mcp_servers_payload(safe.get("MCP_SERVERS") or [])
for key, value in list(safe.items()):
if key in _SECRET_SETTING_KEYS or key in _SETTINGS_DEFAULTS:

4
ouroboros/gateway/tasks.py
View file

@ -9,7 +9,7 @@ import subprocess
import time
import uuid
from datetime import datetime, timezone
from typing import Any, Dict, Iterable, List, Optional
from typing import Any, Dict, List, Optional
from starlette.requests import Request
from starlette.responses import FileResponse, JSONResponse, StreamingResponse
@ -41,7 +41,7 @@ from ouroboros.task_status import (
load_effective_task_result,
)
from ouroboros.tool_access import path_is_relative_to, paths_overlap_casefold
from ouroboros.utils import iter_jsonl_objects, utc_now_iso
from ouroboros.utils import iter_jsonl_objects
from ouroboros.workspace_preflight import (
collect_workspace_preflight,
render_workspace_preflight_summary,

48
ouroboros/gateway/ws.py
View file

@ -28,6 +28,48 @@ def set_event_loop(loop: asyncio.AbstractEventLoop | None) -> None:
_event_loop = loop
_MAX_NATIVE_IMAGE_BYTES = 8 * 1024 * 1024
def _first_image_attachment(attachments: Any) -> tuple[str, str, str]:
"""Resolve the first image upload into (base64, mime, caption).
Attachments reference files already stored by /api/chat/upload under
data/uploads/ — only a validated basename is accepted (no traversal), so
the WS frame cannot read arbitrary paths. Non-image/missing/oversized
attachments keep the text-label-only behavior.
"""
if not isinstance(attachments, list):
return "", "", ""
import base64
import os as _os
uploads_dir = (pathlib.Path(DATA_DIR) / "uploads").resolve(strict=False)
for item in attachments:
if not isinstance(item, dict):
continue
mime = str(item.get("mime") or "").lower()
if not mime.startswith("image/"):
continue
name = _os.path.basename(str(item.get("filename") or "").strip())
if not name or name in {".", ".."}:
continue
path = (uploads_dir / name).resolve(strict=False)
try:
path.relative_to(uploads_dir)
except ValueError:
continue
try:
if not path.is_file() or path.stat().st_size > _MAX_NATIVE_IMAGE_BYTES:
continue
encoded = base64.b64encode(path.read_bytes()).decode("ascii")
except OSError:
continue
caption = str(item.get("display_name") or name)
return encoded, mime, f"[user attachment: {caption}]"
return "", "", ""
def has_ws_clients() -> bool:
with _ws_lock:
return bool(_ws_clients)
@ -219,10 +261,16 @@ async def ws_endpoint(websocket: WebSocket) -> None:
bridge = get_bridge()
if msg_type == "chat":
force_plan = bool(msg.get("force_plan"))
image_b64, image_mime, image_caption = _first_image_attachment(
msg.get("attachments")
)
bridge.ui_send(
payload,
sender_session_id=str(msg.get("sender_session_id", "") or ""),
client_message_id=str(msg.get("client_message_id", "") or ""),
image_base64=image_b64,
image_mime=image_mime,
image_caption=image_caption,
task_metadata={
"force_plan": force_plan,
"force_plan_source": "consilium" if force_plan else "",

6
ouroboros/gateways/claude_code.py
View file

@ -662,6 +662,12 @@ def run_readonly(
def _main() -> int:
if len(sys.argv) >= 2 and sys.argv[1] == "--readonly-child":
try:
from ouroboros.process_custody import start_parent_lifeline
start_parent_lifeline(label="claude-readonly-child")
except Exception:
pass
try:
payload = json.loads(sys.stdin.read() or "{}")
except Exception as exc:

6
ouroboros/headless.py
View file

@ -14,7 +14,6 @@ import shutil
import subprocess
import tempfile
import threading
import time
from datetime import datetime, timezone
from hashlib import sha256
from typing import Any, BinaryIO, Dict, Iterable, List, Optional, Sequence, Tuple
@ -45,7 +44,10 @@ ARTIFACT_TERMINAL_STATUSES = {
ARTIFACT_STATUS_FAILED,
}
_FINAL_STATUSES = {"completed", "failed", "cancelled", "rejected_duplicate"}
# Mirrors task_status.SETTLED_STATUSES; a module-level import would close the
# headless → task_status → outcomes → headless cycle, and the smoke test below
# pins equality so the literal cannot drift from the SSOT.
_FINAL_STATUSES = frozenset({"completed", "failed", "cancelled", "rejected_duplicate"})
_ARTIFACT_LIFECYCLE_FIELDS = {
"artifact_status",
"artifact_error",

8
ouroboros/launcher_bootstrap.py
View file

@ -150,9 +150,11 @@ def load_repo_manifest(repo_dir: pathlib.Path) -> dict[str, Any]:
def _write_repo_manifest(repo_dir: pathlib.Path, manifest: dict[str, Any]) -> None:
meta_path = _managed_meta_path(repo_dir)
meta_path.parent.mkdir(parents=True, exist_ok=True)
meta_path.write_text(json.dumps(manifest, ensure_ascii=True, indent=2) + "\n", encoding="utf-8")
# Atomic: a torn manifest makes the next launch misdetect the installed
# repo generation and re-bootstrap over a healthy checkout.
from ouroboros.utils import atomic_write_json
atomic_write_json(_managed_meta_path(repo_dir), manifest, trailing_newline=True)
def _mark_bootstrap_pin_pending(repo_dir: pathlib.Path) -> None:

85
ouroboros/llm.py
View file

@ -11,7 +11,7 @@ import time
import copy
from typing import Any, Dict, List, Optional, Set, Tuple
from ouroboros.provider_models import normalize_anthropic_model_id, normalize_model_identity
from ouroboros.provider_models import PROVIDER_PREFIXES, normalize_anthropic_model_id, normalize_model_identity
from ouroboros.utils import in_worker_process
log = logging.getLogger(__name__)
@ -26,11 +26,19 @@ class LocalContextTooLargeError(RuntimeError):
def _estimate_message_chars(messages: List[Dict[str, Any]]) -> int:
from ouroboros.context_budget import IMAGE_BLOCK_CHAR_EQUIVALENT
total = 0
for msg in messages:
content = msg.get("content")
if isinstance(content, list):
total += sum(len(str(block.get("text", ""))) for block in content if isinstance(block, dict))
for block in content:
if not isinstance(block, dict):
continue
if str(block.get("type") or "") in ("image_url", "image"):
total += IMAGE_BLOCK_CHAR_EQUIVALENT
continue
total += len(str(block.get("text", "")))
else:
total += len(str(content or ""))
return total
@ -263,11 +271,20 @@ class LLMClient:
resp.status_code,
)
return
from ouroboros.provider_models import update_vision_overlay
for m in resp.json().get("data", []) or []:
mid = m.get("id") or ""
sp = m.get("supported_parameters")
if mid and isinstance(sp, list) and sp:
cls._SUPPORTED_PARAMS_CACHE[mid] = set(sp)
# Vision overlay for supports_vision(): authoritative
# input_modalities from the same /models payload.
arch = m.get("architecture")
if mid and isinstance(arch, dict):
modalities = arch.get("input_modalities")
if isinstance(modalities, list) and modalities:
update_vision_overlay(mid, "image" in modalities)
except Exception:
log.debug("Failed to fetch OpenRouter model capabilities", exc_info=True)
@ -358,14 +375,7 @@ class LLMClient:
@staticmethod
def _parse_provider_model(model: str) -> Tuple[str, str]:
model_name = str(model or "").strip()
for prefix, provider in (
("openai::", "openai"),
("anthropic::", "anthropic"),
("cloudru::", "cloudru"),
("gigachat::", "gigachat"),
("openai-compatible::", "openai-compatible"),
("openrouter::", "openrouter"),
):
for prefix, provider in PROVIDER_PREFIXES:
if model_name.startswith(prefix):
return provider, model_name[len(prefix):].strip()
return "openrouter", model_name
@ -616,6 +626,11 @@ class LLMClient:
block["cache_control"] = {"type": "ephemeral"}
else:
block.pop("cache_control", None)
# Internal metadata (image eviction captions/paths)
# never leaves the process — strict providers 400 on
# unknown content-block fields.
for key in [k for k in block if str(k).startswith("_")]:
block.pop(key, None)
return cleaned
@staticmethod
@ -835,6 +850,7 @@ class LLMClient:
tool_choice,
temperature,
no_proxy,
timeout,
)
if target.get("provider") == "gigachat":
# The gigachat library client is synchronous; offload to a thread
@ -881,6 +897,10 @@ class LLMClient:
kwargs = self._build_remote_kwargs(
target, messages, reasoning_effort, max_tokens, tool_choice, temperature, tools
)
if timeout and timeout > 0:
# Cached clients are built without a timeout; honor the caller's
# per-request timeout instead of silently using the SDK default.
kwargs["timeout"] = float(timeout)
prompt_cache_ttl = self._prompt_cache_ttl_from_payload(
kwargs.get("messages"),
kwargs.get("tools"),
@ -955,6 +975,16 @@ class LLMClient:
flatten_tool_content_blocks=True,
)
)
# Local llama.cpp lane has no vision: replace image blocks with an
# explicit placeholder (a raw image_url block would be str()-flattened
# into the prompt as base64 noise).
for msg in clean_messages:
content = msg.get("content")
if not isinstance(content, list):
continue
for idx, block in enumerate(content):
if isinstance(block, dict) and str(block.get("type") or "") in ("image_url", "image"):
content[idx] = {"type": "text", "text": "[image omitted: model has no vision]"}
local_max = min(max_tokens, 2048)
ctx_len = 0
try:
@ -1503,7 +1533,15 @@ class LLMClient:
sent = session.post(url, headers=headers, json=candidate, timeout=request_timeout)
else:
sent = requests.post(url, headers=headers, json=candidate, timeout=request_timeout)
sent.raise_for_status()
if sent.status_code >= 400:
# Include the Anthropic error body: requests' bare HTTPError text
# ("400 Client Error … for url …") carries no parameter names, so
# the sampling-reject retry matcher could never fire on this path.
body_preview = (sent.text or "")[:2000]
raise requests.HTTPError(
f"{sent.status_code} {sent.reason} for url {sent.url}: {body_preview}",
response=sent,
)
return sent
try:
@ -1573,6 +1611,13 @@ class LLMClient:
parts: List[str] = []
for block in content:
if isinstance(block, dict):
if str(block.get("type") or "") in ("image_url", "image"):
# Explicit placeholder instead of a silent drop: the
# model (and the transcript reader) must know an image
# was present but not deliverable on this lane.
caption = str(block.get("_caption") or "").strip()
parts.append(f"[image omitted: model has no vision{f' — {caption}' if caption else ''}]")
continue
parts.append(str(block.get("text", "")))
else:
parts.append(str(block))
@ -1843,9 +1888,13 @@ class LLMClient:
) -> Dict[str, Any]:
messages = self._normalize_system_message_placement(messages)
resolved_model = str(target.get("resolved_model") or "")
token_limit_key = "max_tokens"
if str(target.get("provider") or "") == "openai" and resolved_model.startswith("gpt-5"):
token_limit_key = "max_completion_tokens"
provider = str(target.get("provider") or "")
# OpenAI reasoning models (gpt-5*, o-series) reject legacy max_tokens
# with a deterministic 400 — they require max_completion_tokens.
openai_reasoning_model = provider == "openai" and resolved_model.startswith(
("gpt-5", "o1", "o3", "o4")
)
token_limit_key = "max_completion_tokens" if openai_reasoning_model else "max_tokens"
if not target.get("supports_openrouter_extensions"):
# Non-OpenRouter providers do not accept cache_control.
clean_messages = self._strip_openrouter_roundtrip_metadata(
@ -1860,6 +1909,10 @@ class LLMClient:
"messages": clean_messages,
token_limit_key: max_tokens,
}
if openai_reasoning_model:
# Direct-OpenAI route honors the configured OUROBOROS_EFFORT_*
# lanes instead of silently dropping them (OpenRouter parity).
kwargs["reasoning_effort"] = normalize_reasoning_effort(reasoning_effort)
if temperature is not None:
kwargs["temperature"] = temperature
if tools:
@ -2111,6 +2164,10 @@ class LLMClient:
kwargs = self._build_remote_kwargs(
target, messages, reasoning_effort, max_tokens, tool_choice, temperature, tools
)
if timeout and timeout > 0:
# Cached clients are built without a timeout; honor the caller's
# per-request timeout instead of silently using the SDK default.
kwargs["timeout"] = float(timeout)
prompt_cache_ttl = self._prompt_cache_ttl_from_payload(
kwargs.get("messages"),
kwargs.get("tools"),

16
ouroboros/local_model.py
View file

@ -4,6 +4,7 @@ from __future__ import annotations
import logging
import os
import pathlib
import subprocess
import sys
import threading
@ -11,7 +12,7 @@ import time
from typing import Any, Callable, Dict, Optional
from ouroboros.platform_layer import (
IS_MACOS, IS_WINDOWS, terminate_process_tree, kill_process_tree,
IS_MACOS, terminate_process_tree, kill_process_tree,
subprocess_new_group_kwargs, subprocess_hidden_kwargs,
)
@ -449,6 +450,19 @@ class LocalModelManager:
)
_popen_kwargs.update(subprocess_new_group_kwargs())
self._proc = subprocess.Popen(cmd, **_with_hidden_subprocess(_popen_kwargs))
try:
from ouroboros.config import DATA_DIR
from ouroboros.process_custody import record_process
record_process(
pathlib.Path(DATA_DIR),
pid=self._proc.pid,
cmd=cmd,
purpose="local_model_server",
scope="session",
)
except Exception:
log.debug("local model custody record failed", exc_info=True)
except FileNotFoundError:
self._status = "error"
self._error = "Python executable not found. Cannot start local model server."

92
ouroboros/loop.py
View file

@ -25,9 +25,6 @@ from ouroboros.utils import estimate_tokens
from ouroboros.loop_tool_execution import (
StatefulToolExecutor,
handle_tool_calls,
_truncate_tool_result,
_TOOL_RESULT_LIMITS,
_DEFAULT_TOOL_RESULT_LIMIT,
)
from ouroboros.loop_llm_call import call_llm_with_retry, emit_llm_usage_event, estimate_cost
@ -53,6 +50,8 @@ class _CompactionRoundContext:
def _estimate_messages_chars(messages: List[Dict[str, Any]]) -> int:
"""Estimate mutable transcript size; excludes the static cached system block."""
from ouroboros.context_budget import IMAGE_BLOCK_CHAR_EQUIVALENT
total = 0
for msg in messages:
content = msg.get("content")
@ -61,7 +60,14 @@ def _estimate_messages_chars(messages: List[Dict[str, Any]]) -> int:
elif isinstance(content, list):
for block in content:
if isinstance(block, dict):
# Count whole multipart blocks, including images/cache markers.
if str(block.get("type") or "") in ("image_url", "image"):
# Vision tokens are billed per tile, not per base64
# char: counting the raw payload made ONE image look
# like ~300K tokens and permanently wedged emergency
# compaction.
total += IMAGE_BLOCK_CHAR_EQUIVALENT
continue
# Count whole multipart blocks, including cache markers.
try:
import json as _json2
total += len(_json2.dumps(block, ensure_ascii=False))
@ -210,15 +216,21 @@ def _skill_finalization_message(drive_root: pathlib.Path, llm_trace: Dict[str, A
def _force_plan_completed(llm_trace: Dict[str, Any]) -> bool:
"""True when a reviewed plan_task completed in this trace.
Reads the structured ``plan_review_aggregate`` flag captured from the FULL
tool result at execution time (loop_tool_execution); the old substring
check against the 700-char trace preview could never see the aggregate
marker at the end of a long plan output, wedging Consilium tasks in the
force-plan reminder loop.
"""
for call in llm_trace.get("tool_calls") or []:
if not isinstance(call, dict):
continue
result_text = str(call.get("result") or "")
if (
str(call.get("tool") or "") == "plan_task"
and not bool(call.get("is_error"))
and "## Plan Review Results" in result_text
and "AGGREGATE:" in result_text
and bool(call.get("plan_review_aggregate"))
):
return True
return False
@ -252,14 +264,16 @@ def _check_budget_limits(
task_cost = accumulated_usage.get("cost", 0)
if budget_remaining_usd <= 0:
finish_reason = f"🚫 Task rejected. Total budget exhausted. Please increase TOTAL_BUDGET in settings."
finish_reason = "🚫 Task rejected. Total budget exhausted. Please increase TOTAL_BUDGET in settings."
accumulated_usage["execution_status"] = "failed"
accumulated_usage["reason_code"] = "budget_exhausted"
return finish_reason, accumulated_usage, llm_trace
budget_pct = task_cost / budget_remaining_usd if budget_remaining_usd > 0 else 1.0
per_task_limit = float(os.environ.get("OUROBOROS_PER_TASK_COST_USD", "20.0") or 20.0)
from ouroboros.config import SETTINGS_DEFAULTS as _DEFAULTS
_per_task_default = str(_DEFAULTS["OUROBOROS_PER_TASK_COST_USD"])
per_task_limit = float(os.environ.get("OUROBOROS_PER_TASK_COST_USD", _per_task_default) or _per_task_default)
if task_cost >= per_task_limit and round_idx % 10 == 0:
_append_or_merge_user_message(
messages,
@ -400,8 +414,52 @@ def _append_or_merge_user_message(messages: List[Dict[str, Any]], text: str) ->
_append_or_merge_user_content(messages, text)
def _evict_stale_image_blocks(messages: List[Dict[str, Any]], *, incoming: int = 0) -> None:
"""Keep only the newest MAX_LIVE_IMAGE_BLOCKS image blocks in the transcript.
Single counter across ALL image sources (owner uploads, browser
screenshots, transport injections). Evicted blocks become a text
placeholder carrying the caption and the re-view path, so the dialogue
HORIZON is preserved while the heavy payload is dropped (P1: granularity
varies, history does not silently vanish). ``incoming`` reserves room for
blocks about to be appended.
"""
from ouroboros.context_budget import MAX_LIVE_IMAGE_BLOCKS
image_refs: List[tuple] = [] # (message_idx, block_idx)
for m_idx, msg in enumerate(messages):
content = msg.get("content")
if not isinstance(content, list):
continue
for b_idx, block in enumerate(content):
if isinstance(block, dict) and str(block.get("type") or "") in ("image_url", "image"):
image_refs.append((m_idx, b_idx))
excess = len(image_refs) + max(0, int(incoming)) - MAX_LIVE_IMAGE_BLOCKS
if excess <= 0:
return
for m_idx, b_idx in image_refs[:excess]:
content = messages[m_idx]["content"]
block = content[b_idx]
caption = str(block.get("_caption") or "").strip()
source_path = str(block.get("_source_path") or "").strip()
placeholder = "[image evicted"
if caption:
placeholder += f": {caption}"
if source_path:
placeholder += f"; re-view: vlm_query file_path={source_path}"
placeholder += "]"
content[b_idx] = {"type": "text", "text": placeholder}
def _append_or_merge_user_content(messages: List[Dict[str, Any]], content: Any) -> None:
"""Append user content without flattening multipart blocks."""
if isinstance(content, list):
incoming_images = sum(
1 for b in content
if isinstance(b, dict) and str(b.get("type") or "") in ("image_url", "image")
)
if incoming_images:
_evict_stale_image_blocks(messages, incoming=incoming_images)
if messages and messages[-1].get("role") == "user":
prior = messages[-1].get("content")
if isinstance(content, list):
@ -747,7 +805,7 @@ def _setup_dynamic_tools(tools_registry, tool_schemas, messages):
parts = []
if enabled:
parts.append(
"✅ Tools are registered in the active v6.17 envelope: "
"✅ Tools are registered in the active capability envelope: "
+ ", ".join(enabled)
)
if not_found:
@ -806,7 +864,7 @@ def _drain_incoming_messages(
break
if drive_root is not None and task_id:
from ouroboros.owner_inject import drain_owner_messages
from ouroboros.owner_mailbox import drain_owner_messages
drive_msgs = drain_owner_messages(drive_root, task_id=task_id, seen_ids=_owner_msg_seen)
for dmsg in drive_msgs:
_append_or_merge_user_message(messages, _owner_marked_content(dmsg))
@ -926,11 +984,13 @@ def run_llm_loop(
tools._ctx.messages = messages
stateful_executor = StatefulToolExecutor()
_owner_msg_seen: set = set()
from ouroboros.config import SETTINGS_DEFAULTS as _DEFAULTS
_max_rounds_default = int(_DEFAULTS["OUROBOROS_MAX_ROUNDS"])
try:
MAX_ROUNDS = max(1, int(os.environ.get("OUROBOROS_MAX_ROUNDS", "200")))
MAX_ROUNDS = max(1, int(os.environ.get("OUROBOROS_MAX_ROUNDS", str(_max_rounds_default))))
except (ValueError, TypeError):
MAX_ROUNDS = 200
log.warning("Invalid OUROBOROS_MAX_ROUNDS, defaulting to 200")
MAX_ROUNDS = _max_rounds_default
log.warning("Invalid OUROBOROS_MAX_ROUNDS, defaulting to %s", _max_rounds_default)
round_idx = 0
try:
while True:
@ -1132,7 +1192,7 @@ def run_llm_loop(
emit_progress(progress_text.strip())
llm_trace["reasoning_notes"].append(progress_text.strip())
error_count = handle_tool_calls(
handle_tool_calls(
tool_calls, tools, drive_logs, task_id, stateful_executor,
messages, llm_trace, emit_progress
)
@ -1173,7 +1233,7 @@ def run_llm_loop(
except Exception:
log.debug("Failed to stop task services", exc_info=True)
try:
from ouroboros.owner_inject import cleanup_task_mailbox
from ouroboros.owner_mailbox import cleanup_task_mailbox
cleanup_task_mailbox(drive_root, task_id)
except Exception:
log.debug("Failed to cleanup task mailbox", exc_info=True)

3
ouroboros/loop_llm_call.py
View file

@ -8,7 +8,6 @@ Extracted from loop.py to keep the main loop orchestrator focused.
from __future__ import annotations
import json
import pathlib
import queue
import time
@ -421,7 +420,6 @@ def call_llm_with_retry(
(None, 0.0) on failure after max_retries
"""
msg = None
last_error: Optional[Exception] = None
drive_root = pathlib.Path(drive_logs).parent
execution_id = str(accumulated_usage.setdefault("execution_id", new_execution_id()))
round_id = f"{execution_id}:round:{round_idx}"
@ -649,7 +647,6 @@ def call_llm_with_retry(
return msg, cost
except Exception as e:
last_error = e
if _record_llm_call_error(
e,
_LlmErrorContext(

5
ouroboros/loop_tool_execution.py
View file

@ -299,6 +299,11 @@ def _extract_result_metadata(fn_name: str, result: Any, is_error: bool) -> Dict[
# late ARTIFACT_OUTPUTS marker (e.g. a stopped service after a long log tail).
if not is_error and "ARTIFACT_OUTPUTS" in text:
meta["artifact_registered"] = True
# Same full-result capture for the Consilium force-plan gate: the review
# aggregate marker sits at the END of a long plan_task result, far past the
# 700-char trace preview the gate used to substring-match against.
if fn_name == "plan_task" and not is_error and "## Plan Review Results" in text and "AGGREGATE:" in text:
meta["plan_review_aggregate"] = True
exit_match = _EXIT_CODE_RE.search(text)
if exit_match:
try:

12
ouroboros/marketplace/fetcher.py
View file

@ -105,6 +105,10 @@ def _validate_member_path(name: str) -> pathlib.PurePosixPath:
parts = posix.parts
_reject_if(any(part == ".." for part in parts), f"Archive member uses '..' traversal: {name!r}")
_reject_if(any(part.startswith("/") for part in parts), f"Archive member has malformed segment: {name!r}")
# Windows drive/ADS escapes: a "C:" segment re-roots pathlib joins on
# Windows and ":" opens NTFS alternate data streams.
_reject_if(any(":" in part or "\\" in part for part in parts),
f"Archive member contains ':' or backslash segment: {name!r}")
return posix
@ -210,6 +214,14 @@ def stage(
file_count += 1
_reject_if(file_count > _MAX_FILE_COUNT, f"Archive exceeds file count cap {_MAX_FILE_COUNT}")
target_path = staging_dir / pathlib.Path(*rel_path.parts)
# Post-join containment: belt-and-braces against any
# platform-specific join surprise the lexical checks missed.
_reject_if(
not target_path.resolve(strict=False).is_relative_to(
staging_dir.resolve(strict=False)
),
f"Archive member escapes staging dir after join: {rel_path}",
)
target_path.parent.mkdir(parents=True, exist_ok=True)
with zf.open(member, "r") as src:
# Zip-bomb defense: trust actual cap+1 read, not file_size metadata.

7
ouroboros/marketplace/isolated_deps.py
View file

@ -218,10 +218,15 @@ def read_deps_state(
return state
fingerprint = read_json_dict(isolated_env_dir(skill_dir) / FINGERPRINT_FILENAME) or {}
if str(state.get("status") or "") != "installed":
# The payload-resident fingerprint.json is AGENT-WRITABLE (it lives in
# the skill dir): it may only ever corroborate a durable deps.json
# record, never substitute for one. A skill shipping a forged
# "installed" fingerprint without the runtime-side install record
# stays non-executable.
if str(fingerprint.get("status") or "") == "installed":
state_hash = str(state.get("specs_hash") or "")
fingerprint_hash = str(fingerprint.get("specs_hash") or "")
if not state_hash or state_hash == fingerprint_hash:
if state_hash and state_hash == fingerprint_hash:
return fingerprint
return state
state_hash = str(state.get("specs_hash") or "")

25
ouroboros/marketplace/ouroboroshub.py
View file

@ -38,10 +38,25 @@ def _raise_if(condition: bool, message: str) -> None:
raise OuroborosHubError(message)
_OPENER = urllib.request.build_opener(AllowlistRedirectHandler(
_ALLOWED_HOSTS,
lambda target: urllib.error.URLError(f"OuroborosHub redirect host {target!r} is not allowed"),
))
# Lazy, proxy-free opener: an import-time build_opener snapshots the process
# proxy environment (and triggers macOS proxy lookup in forked workers); the
# clawhub module's lazy no-proxy pattern is the SSOT behavior to match.
_OPENER: urllib.request.OpenerDirector | None = None
def _hub_opener() -> urllib.request.OpenerDirector:
global _OPENER
if _OPENER is None:
_OPENER = urllib.request.build_opener(
urllib.request.ProxyHandler({}),
AllowlistRedirectHandler(
_ALLOWED_HOSTS,
lambda target: urllib.error.URLError(
f"OuroborosHub redirect host {target!r} is not allowed"
),
),
)
return _OPENER
@dataclass
@ -87,7 +102,7 @@ def _fetch_bytes(url: str, *, max_bytes: int, timeout_sec: int = 15) -> bytes:
_raise_if(parsed.scheme not in {"https", "http"}, f"URL must use https:// (or localhost http): {url}")
_raise_if(parsed.scheme == "http" and parsed.hostname not in {"localhost", "127.0.0.1"}, f"URL must use https:// for non-localhost hosts: {url}")
_raise_if(parsed.hostname not in _ALLOWED_HOSTS, f"Host {parsed.hostname!r} is not allowed for OuroborosHub")
with _OPENER.open(url, timeout=timeout_sec) as resp: # noqa: S310 - host allowlist above
with _hub_opener().open(url, timeout=timeout_sec) as resp: # noqa: S310 - host allowlist above
data = resp.read(max_bytes + 1)
_raise_if(len(data) > max_bytes, f"Response exceeded {max_bytes} bytes: {url}")
return data

4
ouroboros/mcp_client.py
View file

@ -307,7 +307,7 @@ def _mask_token(value: str) -> str:
def looks_masked_secret(value: Any) -> bool:
text = str(value or "").strip()
return text == "***" or text.endswith("...")
return text in ("***", "***set***") or text.endswith("...")
def _redact_error_text(text: Any, cfg: Optional[MCPServerConfig] = None) -> str:
@ -483,7 +483,7 @@ def _serialize_content_part(item: Any) -> Dict[str, Any]:
def _run_async(coro_factory: Callable[[], Awaitable[Any]], *, join_timeout: Optional[int] = None) -> Any:
"""Run async work from sync code; the factory avoids reusing closed coroutines."""
try:
loop = asyncio.get_running_loop()
asyncio.get_running_loop()
except RuntimeError:
return asyncio.run(coro_factory())

86
ouroboros/memory.py
View file

@ -4,7 +4,6 @@ import json
import logging
import os
import pathlib
import hashlib
from collections import Counter
from typing import Any, Dict, List, Optional
@ -46,12 +45,15 @@ class Memory:
return default
def load_scratchpad_blocks(self) -> List[Dict[str, Any]]:
# Lock the STABLE sidecar (not the data fd): writers atomically replace
# the data file via rename, so an fd-lock on the data inode would
# synchronize against an orphaned inode after a swap.
bp = self.scratchpad_blocks_path()
if not bp.exists():
return []
fd = None
try:
fd = os.open(str(bp), os.O_RDONLY)
fd = os.open(str(bp) + ".lock", os.O_RDONLY | os.O_CREAT, 0o644)
_lock_sh(fd)
data = bp.read_text(encoding="utf-8")
blocks = json.loads(data) if data.strip() else []
@ -104,18 +106,17 @@ class Memory:
if metadata:
new_block["metadata"] = dict(metadata)
# Same stable sidecar lock as load/consolidation; the data file itself
# is replaced atomically so a crash mid-append cannot truncate memory.
fd = None
try:
fd = os.open(str(bp), os.O_RDWR | os.O_CREAT, 0o644)
fd = os.open(str(bp) + ".lock", os.O_RDWR | os.O_CREAT, 0o644)
_lock_ex(fd)
raw = b""
while True:
chunk = os.read(fd, 65536)
if not chunk:
break
raw += chunk
text = raw.decode("utf-8", errors="replace").strip()
try:
text = bp.read_text(encoding="utf-8").strip() if bp.exists() else ""
except OSError:
text = ""
blocks = json.loads(text) if text else []
if not isinstance(blocks, list):
blocks = []
@ -133,11 +134,24 @@ class Memory:
})
blocks = blocks[-_SCRATCHPAD_MAX_BLOCKS:]
os.lseek(fd, 0, os.SEEK_SET)
os.ftruncate(fd, 0)
os.write(fd, json.dumps(blocks, ensure_ascii=False, indent=2).encode("utf-8"))
from ouroboros.utils import atomic_write_json
atomic_write_json(bp, blocks)
except Exception:
# An honest journal (P1): a failed write must be journaled as a
# failure and surfaced to the caller — the old path logged
# block_appended success for a block that was never persisted.
log.error("Failed to append scratchpad block", exc_info=True)
try:
append_jsonl(self.journal_path(), {
"ts": utc_now_iso(),
"type": "block_append_failed",
"source": source,
"block": dict(new_block),
})
except Exception:
log.debug("Failed to journal block_append_failed", exc_info=True)
raise
finally:
if fd is not None:
try:
@ -166,6 +180,17 @@ class Memory:
def regenerate_scratchpad_md(self) -> None:
blocks = self.load_scratchpad_blocks()
if not blocks:
bp = self.scratchpad_blocks_path()
if bp.exists() and bp.stat().st_size > 2:
# Storage exists but did not parse — rendering the default
# "(empty)" scratchpad would mask memory corruption as amnesia.
write_text(
self.scratchpad_path(),
"# Scratchpad\n\n⚠️ scratchpad_blocks.json exists but could not be "
"parsed — working memory storage is corrupt, NOT empty. "
"Inspect/restore the file before appending new blocks.\n",
)
return
write_text(self.scratchpad_path(), self._default_scratchpad())
return
@ -296,28 +321,25 @@ class Memory:
return suffix[-max_entries:] if max_entries < len(suffix) else suffix
def jsonl_generation_signature(self, log_name: str) -> Dict[str, Any]:
path = self.logs_path(log_name)
if not path.exists():
return {}
try:
stat = path.stat()
first = ""
with path.open("r", encoding="utf-8") as handle:
for line in handle:
if line.strip():
first = line.strip()
break
return {
"first_line_sha256": hashlib.sha256(first.encode("utf-8", errors="replace")).hexdigest(),
"size": int(stat.st_size),
}
except OSError:
return {}
from ouroboros.utils import jsonl_generation_signature
def summarize_chat(self, entries: List[Dict[str, Any]]) -> str:
return jsonl_generation_signature(self.logs_path(log_name))
def summarize_chat(self, entries: List[Dict[str, Any]], limit: int = 1000) -> str:
"""Render recent chat entries; never hide a horizon cut silently (P1).
Callers that want the FULL window (e.g. low-context mode passes a huge
tail intent) pass a large ``limit``; when truncation does happen the
output says exactly how many older unconsolidated messages were omitted.
"""
if not entries:
return ""
return "\n".join(self._format_chat_line(e, compact=True) for e in entries[-1000:])
limit = max(1, int(limit))
shown = entries[-limit:]
prefix = ""
if len(entries) > len(shown):
prefix = f"[{len(entries) - len(shown)} older unconsolidated messages omitted]\n"
return prefix + "\n".join(self._format_chat_line(e, compact=True) for e in shown)
@staticmethod
def _format_chat_line(e: Dict[str, Any], *, compact: bool) -> str:

1
ouroboros/observability.py
View file

@ -13,7 +13,6 @@ import json
import os
import pathlib
import re
import time
import uuid
from dataclasses import dataclass, field
from typing import Any, Dict, List, Tuple

2
ouroboros/outcomes.py
View file

@ -435,7 +435,7 @@ def normalize_outcome_axes(result: Dict[str, Any]) -> Dict[str, Any]:
def public_task_result(result: Dict[str, Any], *, include_outcome_axes: bool = True) -> Dict[str, Any]:
"""Project persisted/effective task results onto the v6.17 public contract."""
"""Project persisted/effective task results onto the public task-result contract."""
if not isinstance(result, dict):
return {}

0
ouroboros/owner_inject.py → ouroboros/owner_mailbox.py
View file

34
ouroboros/platform_layer.py
View file

@ -552,6 +552,40 @@ def current_process_group_id() -> int:
return 0
def process_start_time(pid: int) -> str:
"""Best-effort stable start-time token for (pid, start_time) fingerprints.
POSIX: ``ps -o lstart=`` (portable across macOS/Linux); Linux fallback
reads /proc/<pid>/stat field 22 (clock ticks since boot). Windows:
empty string — callers degrade to pid-liveness semantics there.
Returns "" when the pid is gone or the platform offers no stable token.
"""
if pid <= 0:
return ""
if os.name == "nt":
return ""
try:
out = subprocess.run(
["ps", "-o", "lstart=", "-p", str(pid)],
capture_output=True, text=True, timeout=5,
)
text = (out.stdout or "").strip()
if out.returncode == 0 and text:
return text
except Exception:
pass
try:
stat_path = pathlib.Path(f"/proc/{pid}/stat")
if stat_path.exists():
fields = stat_path.read_text(encoding="utf-8", errors="replace").rsplit(")", 1)[-1].split()
# rsplit removed fields 1-2 (pid, comm); starttime is field 22 → index 19 here.
if len(fields) >= 20:
return fields[19]
except Exception:
pass
return ""
def process_command(pid: int) -> str:
"""Return a best-effort command line for a Unix process."""
if IS_WINDOWS:

33
ouroboros/post_task_evolution.py
View file

@ -23,7 +23,6 @@ from __future__ import annotations
import json
import logging
import os
import pathlib
from typing import Any, Dict, Optional
@ -79,8 +78,9 @@ def _counter_due(drive_root: pathlib.Path, k: int) -> bool:
n = 0
n += 1
try:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps({"n": n}), encoding="utf-8")
from ouroboros.utils import atomic_write_json
atomic_write_json(path, {"n": n})
except Exception:
pass
return (n % max(1, k)) == 0
@ -188,12 +188,11 @@ def _write_request(drive_root: pathlib.Path, decision: Dict[str, Any], task: Dic
"origin_task_id": str(task.get("id") or ""),
}
path = drive_root / _REQUEST_REL
path.parent.mkdir(parents=True, exist_ok=True)
# Atomic publish: the supervisor polls every tick, so a partial write must
# never be observable (else it could parse-fail and drop the signal).
tmp = path.with_name(f".{path.name}.tmp")
tmp.write_text(json.dumps(req, ensure_ascii=False, indent=2), encoding="utf-8")
os.replace(str(tmp), str(path))
from ouroboros.utils import atomic_write_json
atomic_write_json(path, req)
def maybe_promote(env: Any, task: Dict[str, Any], reflection_entry: Optional[Dict[str, Any]],
@ -272,7 +271,7 @@ def apply_pending_request(drive_root: Any) -> bool:
return False
from supervisor.queue import evolution_block_reason, start_evolution_campaign
from supervisor.state import load_state, save_state
from supervisor.state import load_state
if evolution_block_reason(): # light runtime mode, etc.
_safe_unlink(path)
@ -283,6 +282,11 @@ def apply_pending_request(drive_root: Any) -> bool:
# never run. Drop the stale request rather than leaking it.
_safe_unlink(path)
return False
if bool(st.get("evolution_mode_enabled")):
# A campaign is already enabled (owner-driven or a previous
# promotion). Activating another would hijack its objective and
# clear its failure counter; leave the request for a later tick.
return False
# Per-window budget floor (V4 envelope): if configured, do not start a
# post-task cycle unless at least that much budget remains.
from ouroboros.config import get_post_task_evolution_budget_usd
@ -326,11 +330,14 @@ def apply_pending_request(drive_root: Any) -> bool:
_write_evolution_campaign(camp)
except Exception:
pass
st = load_state()
st["evolution_mode_enabled"] = True
st["evolution_consecutive_failures"] = 0
st["post_task_autostop"] = True
save_state(st)
from supervisor.state import update_state
def _activate_one_shot(live: dict) -> None:
live["evolution_mode_enabled"] = True
live["evolution_consecutive_failures"] = 0
live["post_task_autostop"] = True
update_state(_activate_one_shot)
_safe_unlink(path)
log.info("post_task_evolution: promotion applied -> gated evolution campaign activated")
return True

11
ouroboros/preflight_runner.py
View file

@ -8,7 +8,7 @@ import shutil
import subprocess
import sys
import tempfile
from typing import List, Optional, Sequence
from typing import Optional, Sequence
DEFAULT_PYTEST_ARGS = ["tests/", "-q", "--tb=line", "--no-header"]
@ -60,6 +60,15 @@ def _copy_untracked(repo_dir: pathlib.Path, worktree: pathlib.Path) -> None:
def _preflight_env(temp_root: pathlib.Path, repo_worktree: pathlib.Path) -> dict:
env = dict(os.environ)
env.pop("OUROBOROS_MANAGED_BY_LAUNCHER", None)
# Scrub secret-class variables: the hermetic preflight pytest must not
# inherit live credentials — a (possibly self-written) test could exfiltrate
# or spend with them. OUROBOROS_* wiring stays (suite needs it).
secret_suffixes = ("_API_KEY", "_TOKEN", "_PASSWORD", "_CREDENTIALS", "_SECRET")
for key in list(env):
if key.startswith("OUROBOROS_"):
continue
if key.endswith(secret_suffixes) or key.startswith("GH_"):
env.pop(key, None)
temp_root = pathlib.Path(temp_root).resolve(strict=False)
repo_worktree = pathlib.Path(repo_worktree).resolve(strict=False)
data_dir = (temp_root / "data").resolve(strict=False)

46
ouroboros/pricing.py
View file

@ -14,7 +14,7 @@ from typing import Any, Dict, Optional, Tuple
import logging
from ouroboros.provider_models import normalize_model_identity
from ouroboros.provider_models import normalize_model_identity, provider_for_model
from ouroboros.utils import utc_now_iso
log = logging.getLogger(__name__)
@ -118,7 +118,15 @@ def get_pricing(*, allow_live_fetch: bool = True) -> Dict[str, Tuple[float, ...]
_live = fetch_openrouter_pricing()
if _live and len(_live) > 5:
_cached_pricing.update(_live)
_pricing_fetched = True
_pricing_fetched = True
else:
# fetch_openrouter_pricing swallows network errors and returns
# {} — keep the flag false so the next call retries instead of
# pricing unknown models at $0 for the process lifetime.
import logging as _log
_log.getLogger(__name__).warning(
"OpenRouter pricing fetch returned no data; will retry on next call"
)
except Exception as e:
import logging as _log
_log.getLogger(__name__).warning("Failed to sync pricing from OpenRouter: %s", e)
@ -178,18 +186,11 @@ def infer_api_key_type(model: str, provider: Optional[str] = None) -> str:
if provider_name in {"local", "openrouter", "openai", "anthropic", "openai-compatible", "cloudru", "gigachat"}:
return provider_name
raw_model = str(model or "").strip()
if raw_model.endswith(" (local)"):
return "local"
if raw_model.startswith("openai::"):
return "openai"
if raw_model.startswith("anthropic::"):
return "anthropic"
if raw_model.startswith("openai-compatible::"):
return "openai-compatible"
if raw_model.startswith("cloudru::"):
return "cloudru"
if raw_model.startswith("gigachat::"):
return "gigachat"
direct_provider = provider_for_model(raw_model)
# ``openrouter::``-prefixed and un-prefixed ids both bill OpenRouter and
# fall through to the normalized-identity heuristics below.
if direct_provider not in ("openrouter",):
return direct_provider
normalized = normalize_model_identity(raw_model)
if normalized.startswith("openai/"):
return "openrouter"
@ -220,20 +221,9 @@ def infer_provider_from_model(model: str) -> str:
Used by review-pipeline emitters to ensure /api/cost-breakdown attribution
is correct regardless of which provider the model actually routes through.
"""
raw = str(model or "").strip()
if raw.endswith(" (local)"):
raw = raw[:-8]
if raw.startswith("anthropic::"):
return "anthropic"
if raw.startswith("openai::"):
return "openai"
if raw.startswith("openai-compatible::"):
return "openai-compatible"
if raw.startswith("cloudru::"):
return "cloudru"
if raw.startswith("gigachat::"):
return "gigachat"
return "openrouter"
provider = provider_for_model(model)
# Historical billing attribution: local-suffixed ids billed as openrouter.
return "openrouter" if provider == "local" else provider
def infer_model_category(model: str) -> str:

338
ouroboros/process_custody.py Normal file
View file

@ -0,0 +1,338 @@
"""Process custody: supervised spawning plus a durable orphan ledger.
Every long-lived OS process Ouroboros spawns should go through
``spawn_supervised`` so its identity lands in ``data/state/process_ledger.jsonl``
BEFORE it can be orphaned. The reaper (server startup + periodic tick) then
kills ledger entries whose owning generation is gone, matching processes by a
STRICT (pid, start_time, cmd_sha256) fingerprint — never by command-line
class, so a dev instance can never reap a packaged instance's processes (or
vice versa).
Scopes:
- ``task``: dies with its owning task; reapable as soon as the task is no
longer running (and always across server generations).
- ``session``: dies with the server generation (session_id mismatch → reap).
- ``daemon``: intentionally outlives generations (e.g. launcher-managed
companions); the reaper only prunes dead entries, never kills.
This module deliberately lives OUTSIDE platform_layer (primitives-only) and
adds no policy to the panic layers it complements (``_active_subprocesses``,
port sweeps, Windows Job Objects all stay).
"""
from __future__ import annotations
import hashlib
import logging
import os
import pathlib
import subprocess
import uuid
from typing import Any, Dict, List, Optional
from ouroboros.platform_layer import (
kill_process_group_id,
pid_is_alive,
process_command,
process_group_id,
process_start_time,
subprocess_new_group_kwargs,
)
from ouroboros.utils import append_jsonl, utc_now_iso
log = logging.getLogger(__name__)
LEDGER_FILENAME = "process_ledger.jsonl"
# One ledger generation per server process; recorded into every entry so the
# reaper can tell "mine" from "previous generation" without guessing.
_SESSION_ID = uuid.uuid4().hex
_VALID_SCOPES = ("task", "session", "daemon")
def current_custody_session_id() -> str:
return _SESSION_ID
def ledger_path(drive_root: pathlib.Path) -> pathlib.Path:
return pathlib.Path(drive_root) / "state" / LEDGER_FILENAME
def _cmd_sha256(cmd: Any) -> str:
try:
if isinstance(cmd, (list, tuple)):
text = "\0".join(str(part) for part in cmd)
else:
text = str(cmd or "")
return hashlib.sha256(text.encode("utf-8", errors="replace")).hexdigest()
except Exception:
return ""
def _live_cmd_sha256(pid: int) -> str:
command = process_command(pid)
if not command:
return ""
return hashlib.sha256(command.encode("utf-8", errors="replace")).hexdigest()
def record_process(
drive_root: pathlib.Path,
*,
pid: int,
cmd: Any,
purpose: str,
scope: str,
owner_task_id: str = "",
) -> Dict[str, Any]:
"""Append a custody record for an already-spawned process."""
if scope not in _VALID_SCOPES:
raise ValueError(f"process custody scope must be one of {_VALID_SCOPES}, got {scope!r}")
try:
pgid = process_group_id(pid)
except Exception:
pgid = 0
entry = {
"ts": utc_now_iso(),
"pid": int(pid),
"pgid": int(pgid or 0),
"fingerprint": {
"start_time": process_start_time(pid),
# The LIVE command line (what the OS reports) is the reap-time
# comparison anchor; the argv we passed may differ cosmetically.
"cmd_sha256": _live_cmd_sha256(pid) or _cmd_sha256(cmd),
},
"purpose": str(purpose or "")[:200],
"scope": scope,
"owner_task": str(owner_task_id or ""),
"session_id": _SESSION_ID,
}
append_jsonl(ledger_path(drive_root), entry)
return entry
def spawn_supervised(
cmd: Any,
*,
drive_root: pathlib.Path,
purpose: str,
scope: str,
owner_task_id: str = "",
new_process_group: bool = True,
**popen_kwargs: Any,
) -> subprocess.Popen:
"""Popen + durable custody record (the single supervised chokepoint).
The record is written immediately after spawn, so even a SIGKILL of the
spawning worker cannot orphan the child invisibly — the reaper finds it
in the ledger on the next generation.
"""
if new_process_group:
merged = dict(subprocess_new_group_kwargs())
merged.update(popen_kwargs)
popen_kwargs = merged
proc = subprocess.Popen(cmd, **popen_kwargs) # noqa: S603 — callers pass vetted argv lists
try:
record_process(
drive_root,
pid=proc.pid,
cmd=cmd,
purpose=purpose,
scope=scope,
owner_task_id=owner_task_id,
)
except Exception:
log.warning("process custody record failed for pid %s (%s)", proc.pid, purpose, exc_info=True)
return proc
def _fingerprint_matches(entry: Dict[str, Any]) -> bool:
"""STRICT identity: the live process must still BE the recorded one.
pid alive + same start_time (when we have one) + same command hash (when
we have one). A recycled pid fails this and is left alone. We never match
by command-line class.
"""
pid = int(entry.get("pid") or 0)
if pid <= 0 or not pid_is_alive(pid):
return False
fp = entry.get("fingerprint") if isinstance(entry.get("fingerprint"), dict) else {}
recorded_start = str(fp.get("start_time") or "")
if recorded_start:
live_start = process_start_time(pid)
if live_start and live_start != recorded_start:
return False
recorded_cmd = str(fp.get("cmd_sha256") or "")
if recorded_cmd:
live_cmd = _live_cmd_sha256(pid)
if live_cmd and live_cmd != recorded_cmd:
return False
if not live_cmd and not recorded_start:
# Windows: no command line and no start time — liveness is all we
# have (same degradation as workspace_executor records).
return True
return True
def _read_ledger(drive_root: pathlib.Path) -> List[Dict[str, Any]]:
path = ledger_path(drive_root)
if not path.exists():
return []
entries: List[Dict[str, Any]] = []
try:
import json
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
line = line.strip()
if not line:
continue
try:
obj = json.loads(line)
except ValueError:
continue
if isinstance(obj, dict) and obj.get("pid"):
entries.append(obj)
except OSError:
return []
# Last record per pid wins (a pid may be re-registered by a newer spawn).
by_pid: Dict[int, Dict[str, Any]] = {}
for entry in entries:
try:
by_pid[int(entry.get("pid") or 0)] = entry
except (TypeError, ValueError):
continue
by_pid.pop(0, None)
return list(by_pid.values())
def _rewrite_ledger(drive_root: pathlib.Path, entries: List[Dict[str, Any]]) -> None:
import json
path = ledger_path(drive_root)
try:
from ouroboros.utils import jsonl_append_lock_path
from ouroboros.platform_layer import acquire_exclusive_file_lock, release_exclusive_file_lock
lock_path = jsonl_append_lock_path(path)
lock_fd = acquire_exclusive_file_lock(lock_path, timeout_sec=2.0, stale_sec=10.0)
try:
tmp = path.with_name(path.name + f".tmp.{os.getpid()}")
tmp.write_text(
"".join(json.dumps(entry, ensure_ascii=False) + "\n" for entry in entries),
encoding="utf-8",
)
os.replace(tmp, path)
finally:
release_exclusive_file_lock(lock_path, lock_fd)
except Exception:
log.debug("process ledger rewrite failed", exc_info=True)
def start_parent_lifeline(*, poll_sec: float = 5.0, label: str = "") -> None:
"""Daemon watchdog: group-suicide when the parent process dies (POSIX).
For OUR python entrypoints only (workers, extension runner, claude child):
when the parent dies, the child is reparented to init (ppid==1) and would
otherwise keep burning CPU/budget invisibly. Arbitrary-argv services and
skills cannot get a watchdog injected — they are covered by the ledger +
reaper instead.
"""
if os.name == "nt":
return # Windows children are covered by Job Objects
import threading
import time as _time
def _suicide() -> None:
log.warning("parent process died — lifeline group-suicide (%s)", label or "child")
try:
from ouroboros.platform_layer import current_process_group_id
pgid = current_process_group_id()
# Kill the whole group ONLY when we lead it (worker/runner
# entrypoints setsid/new-group first). A non-leader sharing the
# spawner's group must not take unrelated siblings down with it.
if pgid == os.getpid():
kill_process_group_id(pgid)
except Exception:
pass
os._exit(1)
initial_ppid = os.getppid()
if initial_ppid <= 1:
# The parent died before we even got here (import-delay race after an
# abrupt supervisor kill). These entrypoints are always spawned by a
# live Ouroboros parent, so an orphan at startup is already a leak.
_suicide()
return
def _watch() -> None:
while True:
_time.sleep(poll_sec)
# Any reparenting means the original parent died (orphans go to
# init/launchd or the nearest subreaper).
if os.getppid() != initial_ppid:
_suicide()
threading.Thread(target=_watch, daemon=True, name=f"parent-lifeline-{label or 'child'}").start()
def reap_orphaned_processes(
drive_root: pathlib.Path,
*,
running_task_ids: Optional[set] = None,
) -> List[int]:
"""Kill ledgered processes whose owning generation/task is gone.
Rules:
- dead pid / fingerprint mismatch → prune the entry, never kill;
- current session's entries → keep (their owners are alive);
EXCEPT task-scoped entries whose owner task is no longer running;
- previous generations: task/session scopes → kill group + reap event;
daemon scope → keep (launcher-managed lifecycles).
"""
drive_root = pathlib.Path(drive_root)
entries = _read_ledger(drive_root)
if not entries:
return []
reaped: List[int] = []
survivors: List[Dict[str, Any]] = []
for entry in entries:
pid = int(entry.get("pid") or 0)
scope = str(entry.get("scope") or "task")
same_session = str(entry.get("session_id") or "") == _SESSION_ID
if not _fingerprint_matches(entry):
continue # dead or recycled pid: prune silently
owner_task = str(entry.get("owner_task") or "")
task_owner_gone = (
scope == "task"
and running_task_ids is not None
and owner_task
and owner_task not in running_task_ids
)
if scope == "daemon" or (same_session and not task_owner_gone):
survivors.append(entry)
continue
try:
pgid = int(entry.get("pgid") or 0)
if pgid > 0:
kill_process_group_id(pgid)
else:
from ouroboros.platform_layer import kill_pid_tree
kill_pid_tree(pid)
reaped.append(pid)
append_jsonl(drive_root / "logs" / "supervisor.jsonl", {
"ts": utc_now_iso(),
"type": "process_reaped",
"pid": pid,
"pgid": int(entry.get("pgid") or 0),
"purpose": entry.get("purpose"),
"scope": scope,
"owner_task": owner_task,
"stale_session": str(entry.get("session_id") or ""),
})
except Exception:
log.warning("Failed to reap ledgered process %s", pid, exc_info=True)
survivors.append(entry)
_rewrite_ledger(drive_root, survivors)
return reaped

118
ouroboros/provider_models.py
View file

@ -1,7 +1,85 @@
"""Provider-specific model ID helpers and direct-provider defaults."""
"""Provider-specific model ID helpers, direct-provider defaults, and the
provider registry (SSOT for prefix→provider→credentials knowledge that was
previously duplicated across llm.py, pricing.py, agent_task_pipeline.py and
deep_self_review.py)."""
from __future__ import annotations
import os
# Direct-provider prefix → canonical provider name. Un-prefixed models route
# through OpenRouter. Order matters only for readability; prefixes are disjoint.
PROVIDER_PREFIXES: tuple[tuple[str, str], ...] = (
("openai::", "openai"),
("anthropic::", "anthropic"),
("cloudru::", "cloudru"),
("gigachat::", "gigachat"),
("openai-compatible::", "openai-compatible"),
("openrouter::", "openrouter"),
)
# Primary credential env var per provider (single-key providers).
PROVIDER_ENV_KEYS: dict[str, str] = {
"openai": "OPENAI_API_KEY",
"anthropic": "ANTHROPIC_API_KEY",
"cloudru": "CLOUDRU_FOUNDATION_MODELS_API_KEY",
"openrouter": "OPENROUTER_API_KEY",
}
def provider_for_model(model: str) -> str:
"""Return the execution provider for a model id (``local`` for local lanes)."""
name = str(model or "").strip()
if name.endswith(" (local)"):
return "local"
for prefix, provider in PROVIDER_PREFIXES:
if name.startswith(prefix):
return provider
return "openrouter"
def provider_has_credentials(provider: str) -> bool:
"""Return True when the environment carries usable credentials for a provider."""
if provider == "local":
return True
if provider == "openai-compatible":
compat = str(os.environ.get("OPENAI_COMPATIBLE_API_KEY", "") or "").strip()
legacy_key = str(os.environ.get("OPENAI_API_KEY", "") or "").strip()
legacy_base = str(os.environ.get("OPENAI_BASE_URL", "") or "").strip()
return bool(compat or (legacy_key and legacy_base))
if provider == "gigachat":
creds = str(os.environ.get("GIGACHAT_CREDENTIALS", "") or "").strip()
user = str(os.environ.get("GIGACHAT_USER", "") or "").strip()
password = str(os.environ.get("GIGACHAT_PASSWORD", "") or "").strip()
return bool(creds or (user and password))
env_key = PROVIDER_ENV_KEYS.get(provider, "OPENROUTER_API_KEY")
return bool(str(os.environ.get(env_key, "") or "").strip())
def model_has_credentials(model: str) -> bool:
"""Return True when the model's provider has usable credentials configured."""
return provider_has_credentials(provider_for_model(model))
def resolve_credentialed_model(default_model: str) -> str:
"""Return ``default_model`` if its provider is credentialed, else the first
configured model slot whose provider has credentials (light → fallback →
main → code). Falls back to ``default_model`` when nothing is credentialed
so callers surface the original provider error rather than a silent swap."""
if model_has_credentials(default_model):
return default_model
for env_name in (
"OUROBOROS_MODEL_LIGHT",
"OUROBOROS_MODEL_FALLBACK",
"OUROBOROS_MODEL",
"OUROBOROS_MODEL_CODE",
):
candidate = str(os.environ.get(env_name, "") or "").strip()
if candidate and model_has_credentials(candidate):
return candidate
return default_model
OPENAI_DIRECT_DEFAULTS = {
"main": "openai::gpt-5.5",
"code": "openai::gpt-5.5",
@ -97,6 +175,44 @@ def compute_direct_review_models_fallback(
return [main] * int(review_runs or 3)
# Conservative static vision map by normalized id/prefix. The OpenRouter
# /models overlay (llm.py) refines this at runtime; static knowledge only
# covers families whose vision support is long-established.
_VISION_MODEL_PREFIXES: tuple[str, ...] = (
"openai/gpt-5", "openai/gpt-4o", "openai/gpt-4.1", "openai/o3", "openai/o4",
"google/gemini-", "anthropic/claude-",
"x-ai/grok-4", "x-ai/grok-3",
"qwen/qwen-vl", "qwen/qwen2.5-vl", "qwen/qwen3-vl",
"mistralai/pixtral", "meta-llama/llama-4", "meta-llama/llama-3.2-90b-vision",
"openai/gpt-5.5",
)
# Runtime overlay: model_id → bool, fed from OpenRouter /models
# architecture.input_modalities by llm.py (same lifecycle as its
# supported-parameters cache).
_VISION_OVERLAY: dict = {}
def update_vision_overlay(model_id: str, supports: bool) -> None:
normalized = normalize_model_identity(model_id)
if normalized:
_VISION_OVERLAY[normalized] = bool(supports)
def supports_vision(model_id: str) -> bool:
"""True when the model accepts native image input blocks."""
# Local lanes have no vision regardless of family name; check the RAW id —
# normalize_model_identity strips the " (local)" suffix.
if str(model_id or "").strip().endswith(" (local)"):
return False
normalized = normalize_model_identity(model_id)
if not normalized:
return False
if normalized in _VISION_OVERLAY:
return _VISION_OVERLAY[normalized]
return normalized.startswith(_VISION_MODEL_PREFIXES)
def normalize_model_identity(model: str) -> str:
text = str(model or "").strip()
if text.endswith(" (local)"):

10
ouroboros/reflection.py
View file

@ -4,7 +4,6 @@ from __future__ import annotations
import json
import logging
import os
import pathlib
from typing import Any, Dict, List, Optional
@ -533,14 +532,19 @@ def _update_patterns(drive_root: pathlib.Path, entry: Dict[str, Any]) -> None:
else:
current = _PATTERNS_HEADER
current_truncated = _truncate_with_notice(current, 3000)
# The register is bounded by the prompt contract (max 20 rows), which fits
# well under this cap — the old 3000-char cut fed the LLM a PARTIAL table
# and the full-replace write then dropped every unseen row (memory loss).
# The cap remains only as a backstop against a pathologically bloated file.
_register_cap = 16_000
current_truncated = _truncate_with_notice(current, _register_cap)
prompt = _PATTERNS_PROMPT.format(
current_patterns=(
current_truncated
+ (
"\n\n[IMPORTANT: The current register was compacted for prompt size. "
"Preserve existing rows unless you are intentionally merging or updating them.]"
if len(current) > 3000 else ""
if len(current) > _register_cap else ""
)
),
goal=_truncate_with_notice(entry.get("goal", "?"), 200),

13
ouroboros/review_evidence.py
View file

@ -193,6 +193,12 @@ def build_review_projection(
effective_status = matching_run.status if matching_run else ("stale" if latest else "none")
open_obligations = state.get_open_obligations(repo_key=repo_filter)
open_debts = state.get_open_commit_readiness_debts(repo_key=repo_filter)
try:
from ouroboros.utils import read_json_dict
advisory_overrides = read_json_dict(drive_root_path / "state" / "advisory_overrides.json") or {}
except Exception:
advisory_overrides = {}
return {
"state": state,
"filters": {
@ -223,6 +229,7 @@ def build_review_projection(
"open_debts": open_debts,
"repo_commit_ready": bool(effective_is_fresh and not open_obligations and not open_debts),
"retry_anchor": "commit_readiness_debt" if open_debts else None,
"advisory_overrides": advisory_overrides,
}
@ -250,6 +257,12 @@ def build_review_status_payload(projection: Dict[str, Any], *, next_step: str, i
"next_step": next_step,
}
payload["message"] = payload["status_summary"]
# Persistent advisory-enforcement visibility (BIBLE P3 loud-advisory bound):
# how many blocking-grade signals advisory enforcement waved through.
overrides = projection.get("advisory_overrides")
if isinstance(overrides, dict) and overrides.get("count"):
payload["advisory_overrides_count"] = int(overrides.get("count") or 0)
payload["advisory_overrides_recent"] = list(overrides.get("recent") or [])
if include_raw and selected_attempt is not None:
payload["raw_evidence"] = {
"attempt_ts": selected_attempt.ts,

11
ouroboros/review_state.py
View file

@ -1129,12 +1129,17 @@ def _save_state_unlocked(drive_root: pathlib.Path, state: AdvisoryReviewState) -
def save_state(drive_root: pathlib.Path, state: AdvisoryReviewState) -> None:
"""Persist review state atomically under a best-effort lock."""
"""Persist review state atomically under the review-state lock.
Raises ``TimeoutError`` on lock failure (matching ``update_state``): a
silently skipped save left the advisory ledger reporting a stale "fresh"
pre-review, which the commit gate then trusted — an immune-system hole,
not a tolerable degradation.
"""
lock_path = drive_root / _LOCK_RELPATH
lock_fd = acquire_review_state_lock(drive_root)
if lock_fd is None:
log.warning("Failed to acquire review state lock at %s; skipping save", lock_path)
return
raise TimeoutError(f"Could not acquire review state lock for {lock_path}")
try:
_save_state_unlocked(drive_root, state)
finally:

2
ouroboros/review_substrate.py
View file

@ -21,7 +21,7 @@ from ouroboros.config import get_review_models
from ouroboros.llm import LLMClient
from ouroboros.observability import new_call_id, persist_call
from ouroboros.triad_review import extract_json_array
from ouroboros.utils import sanitize_tool_result_for_log, truncate_review_artifact, utc_now_iso
from ouroboros.utils import sanitize_tool_result_for_log, truncate_review_artifact
@dataclass(frozen=True)

4
ouroboros/safety.py
View file

@ -143,8 +143,10 @@ TOOL_POLICY: Dict[str, str] = {
# run_command safe-subject whitelist.
# ``pip`` mutates the Python env and must route through the LLM check.
# ``find`` is NOT safe: -delete / -exec rm make it a mutator, so it routes
# through the LLM safety check like other mutating commands.
SAFE_SHELL_COMMANDS = frozenset([
"ls", "cat", "head", "tail", "grep", "rg", "find", "wc",
"ls", "cat", "head", "tail", "grep", "rg", "wc",
"git", "pytest", "pwd", "whoami",
"date", "which", "file", "stat", "diff", "tree",
"du", "df",

79
ouroboros/server_auth.py
View file

@ -8,6 +8,9 @@ import hmac
import html
import ipaddress
import os
import pathlib
import secrets
import time
from http.cookies import SimpleCookie
from urllib.parse import parse_qs
@ -69,8 +72,67 @@ def _headers_map(scope: Scope) -> dict[str, str]:
}
def _session_value(password: str) -> str:
return hashlib.sha256(f"ouroboros-auth\0{password}".encode("utf-8")).hexdigest()
# Sessions expire; a leaked cookie is no longer a permanent credential.
_AUTH_SESSION_TTL_SEC = 30 * 24 * 3600
_auth_secret_cache: bytes | None = None
def _auth_secret() -> bytes:
"""Server-side random HMAC key, persisted across restarts (0600).
The old cookie was a bare ``sha256(password)`` — deterministic, never
expiring, and forgeable by anyone who ever saw it. Sessions are now
HMAC-signed with this key, so they cannot be minted without server state
and they die with the key file (one re-login after key loss is accepted).
"""
global _auth_secret_cache
if _auth_secret_cache:
return _auth_secret_cache
try:
from ouroboros import config as _cfg
path = pathlib.Path(_cfg.DATA_DIR) / "state" / "auth_secret.key"
path.parent.mkdir(parents=True, exist_ok=True)
if path.exists():
text = path.read_text(encoding="utf-8").strip()
if text:
_auth_secret_cache = bytes.fromhex(text)
return _auth_secret_cache
secret = secrets.token_hex(32)
path.write_text(secret + "\n", encoding="utf-8")
try:
os.chmod(path, 0o600)
except OSError:
pass
_auth_secret_cache = bytes.fromhex(secret)
except Exception:
# No durable key (read-only data dir): fall back to a process-lifetime
# key — sessions survive within the run and re-login after restart.
_auth_secret_cache = secrets.token_bytes(32)
return _auth_secret_cache
def _session_value(password: str, issued_at: int | None = None) -> str:
issued = int(issued_at if issued_at is not None else time.time())
pw_digest = hashlib.sha256(f"ouroboros-auth\0{password}".encode("utf-8")).hexdigest()
signature = hmac.new(
_auth_secret(), f"{issued}\0{pw_digest}".encode("utf-8"), hashlib.sha256
).hexdigest()
return f"{issued}.{signature}"
def _cookie_session_valid(cookie_val: str, password: str) -> bool:
issued_text, sep, _signature = (cookie_val or "").partition(".")
if not sep:
return False
try:
issued = int(issued_text)
except (TypeError, ValueError):
return False
now = time.time()
if issued > now + 300 or now - issued > _AUTH_SESSION_TTL_SEC:
return False
return hmac.compare_digest(cookie_val, _session_value(password, issued))
def _cookie_value(scope: Scope, cookie_name: str) -> str:
@ -107,7 +169,7 @@ def _candidate_password(scope: Scope) -> str:
def _is_authenticated(scope: Scope, password: str) -> bool:
cookie_val = _cookie_value(scope, AUTH_COOKIE_NAME)
if cookie_val and hmac.compare_digest(cookie_val, _session_value(password)):
if cookie_val and _cookie_session_valid(cookie_val, password):
return True
candidate = _candidate_password(scope)
return bool(candidate) and hmac.compare_digest(candidate, password)
@ -212,7 +274,16 @@ async def _handle_login(scope: Scope, receive: Receive, send: Send, password: st
response = JSONResponse({"ok": True, "next": next_url}, status_code=200)
else:
response = RedirectResponse(next_url, status_code=303)
response.set_cookie(AUTH_COOKIE_NAME, _session_value(password), httponly=True, samesite="lax")
# Secure only when actually served over TLS: setting it on plain-http LAN
# would make the browser drop the cookie and lock the user in a login loop.
response.set_cookie(
AUTH_COOKIE_NAME,
_session_value(password),
max_age=_AUTH_SESSION_TTL_SEC,
httponly=True,
samesite="lax",
secure=str(scope.get("scheme") or "").lower() in {"https", "wss"},
)
await response(scope, receive, send)

10
ouroboros/server_control.py
View file

@ -119,8 +119,14 @@ def execute_panic_stop(
force_kill_pid(child.pid)
except (ProcessLookupError, PermissionError):
pass
kill_process_on_port(8765)
kill_process_on_port(8766)
# Sweep the actually bound main port (not hardcoded 8765/8766 — a
# custom-port install would panic-kill an unrelated listener).
try:
import server as _server_mod
kill_process_on_port(_server_mod._actual_bound_port())
except Exception:
kill_process_on_port(8765)
kill_process_on_port(host_service_port())
except Exception:
pass

57
ouroboros/skill_lifecycle_queue.py
View file

@ -6,6 +6,7 @@ import asyncio
import contextlib
import logging
import os
import time
import pathlib
import re
import threading
@ -136,12 +137,36 @@ def _notify_chat_progress(job: LifecycleJob, phase: str) -> None:
return
def _lifecycle_deadline_sec() -> float:
"""Upper bound for waiting on the lifecycle lane / running one job.
A single wedged job used to freeze the whole skill-lifecycle queue forever
(unbounded join + infinite lock-acquire loops); the deadline converts that
silent wedge into a loud, attributable failure. Generous default — skill
reviews legitimately run for many minutes.
"""
raw = os.environ.get("OUROBOROS_SKILL_LIFECYCLE_TIMEOUT_SEC", "")
try:
parsed = float(raw)
if parsed > 0:
return parsed
except (TypeError, ValueError):
pass
return 1800.0
@contextlib.asynccontextmanager
async def _async_thread_lock(lock: threading.Lock):
deadline = time.monotonic() + _lifecycle_deadline_sec()
acquired = False
while not acquired:
acquired = lock.acquire(blocking=False)
if not acquired:
if time.monotonic() >= deadline:
raise TimeoutError(
"Skill lifecycle lane is busy beyond the deadline — a previous "
"job appears wedged. Restarting Ouroboros recovers the lane."
)
await asyncio.sleep(0.01)
try:
yield
@ -192,6 +217,7 @@ async def async_skill_lifecycle_file_lock(drive_root: pathlib.Path):
lock_dir = pathlib.Path(drive_root) / "state"
lock_dir.mkdir(parents=True, exist_ok=True)
lock_path = lock_dir / "skill_lifecycle.lock"
deadline = time.monotonic() + _lifecycle_deadline_sec()
with lock_path.open("a+") as fh:
acquired = False
while not acquired:
@ -199,6 +225,11 @@ async def async_skill_lifecycle_file_lock(drive_root: pathlib.Path):
file_lock_exclusive_nb(fh.fileno())
acquired = True
except OSError:
if time.monotonic() >= deadline:
raise TimeoutError(
"skill_lifecycle.lock is held beyond the deadline — a "
"previous job appears wedged. Restarting Ouroboros recovers it."
)
await asyncio.sleep(0.05)
try:
yield
@ -312,6 +343,22 @@ async def run_lifecycle_job(
if not terminal_notified:
_notify_chat(job)
raise
except BaseException as exc:
# Lane-acquisition failures (e.g. the new _async_thread_lock deadline
# TimeoutError) raise BEFORE the inner finally exists — without this
# branch the job stayed "queued" in _dedupe_jobs forever, blocking
# every future job with the same dedupe key.
if not terminal_notified:
job.status = "failed"
job.error = job.error or (str(exc) or type(exc).__name__)
job.finished_at = job.finished_at or _now_iso()
_release_dedupe(job)
if _active is job:
_active = None
if opts.progress_target is not None:
opts.progress_target.release()
_notify_chat(job)
raise
async def run_blocking_preserving_cancellation(
@ -393,7 +440,15 @@ def run_lifecycle_job_blocking(
thread = threading.Thread(target=_thread_main, name=f"skill-lifecycle-{kind}", daemon=False)
thread.start()
thread.join()
thread.join(timeout=_lifecycle_deadline_sec())
if thread.is_alive():
# The job thread is wedged. We cannot kill it safely (it may hold the
# cross-process flock, which the OS releases only on process exit), but
# the caller must not hang forever pretending the job is progressing.
raise TimeoutError(
f"Skill lifecycle job '{kind}:{target}' exceeded its deadline and is "
"wedged; the lifecycle lane stays blocked until restart."
)
if "error" in box:
raise box["error"]
return box.get("result")

3
ouroboros/skill_loader.py
View file

@ -9,11 +9,8 @@ pending review. Scripts execute via ``skill_exec``; extensions load through
from __future__ import annotations
import hashlib
import json
import logging
import pathlib
import re
import tempfile
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional

1
ouroboros/skill_review.py
View file

@ -215,7 +215,6 @@ def _build_skill_file_pack(
if len(files) > _MAX_SKILL_FILES:
# Never truncate the executable review surface.
raise _SkillPackTooLarge(len(files), _MAX_SKILL_FILES)
extras = 0
blocks: List[str] = []
for file_path in files:

5
ouroboros/skill_review_runner.py
View file

@ -17,19 +17,14 @@ from ouroboros.skill_lifecycle_queue import (
LifecycleJob,
LifecycleJobOptions,
run_blocking_preserving_cancellation,
run_lifecycle_job,
run_lifecycle_job_blocking,
)
from ouroboros.skill_loader import (
SkillPayloadUnreadable,
SkillReviewState,
compute_content_hash,
find_skill,
requested_core_setting_keys,
review_status_allows_execution,
save_enabled,
save_review_state,
save_skill_grants,
skill_review_gate,
skill_state_dir,
)

67
ouroboros/task_results.py
View file

@ -7,7 +7,7 @@ import pathlib
import re
from typing import Any, Dict, List, Optional
from ouroboros.utils import atomic_write_json, read_json_dict, utc_now_iso
from ouroboros.utils import atomic_write_json, read_json_dict, update_json_locked, utc_now_iso
log = logging.getLogger(__name__)
@ -137,30 +137,47 @@ def write_task_result(
status: str,
**fields: Any,
) -> Dict[str, Any]:
"""Merge-write a task result under a per-file lock.
Worker processes, the supervisor thread, and gateway handlers all
read-modify-write the same ``task_results/<id>.json``; the lock makes the
monotonic-status guard evaluate the CURRENT on-disk status (closing the
"cancel_requested latch erased by a concurrent completed write" window).
"""
path = task_result_path(results_drive_root, task_id)
existing = load_task_result(results_drive_root, task_id) or {}
explicit_ts = str(fields.pop("ts", "") or "")
# Monotonic lifecycle: never let a stale scheduled/running mirror overwrite a
# cancel-intent latch or a terminal outcome. This is the structural guard
# against "ghost" tasks that keep reporting scheduled/running after they were
# cancelled or finished.
if existing and _is_status_regression(existing.get("status"), status):
# Surface the blocked transition: when debugging a "stuck" task this is
# the only signal that a stale/late write was intentionally dropped.
log.debug("Blocked status regression %s -> %s for task %s",
existing.get("status"), status, task_id)
return existing
def _merge(existing: Dict[str, Any]) -> Optional[Dict[str, Any]]:
# Monotonic lifecycle: never let a stale scheduled/running mirror
# overwrite a cancel-intent latch or a terminal outcome. This is the
# structural guard against "ghost" tasks that keep reporting
# scheduled/running after they were cancelled or finished.
if existing and _is_status_regression(existing.get("status"), status):
# Surface the blocked transition: when debugging a "stuck" task this
# is the only signal that a stale/late write was intentionally dropped.
log.debug("Blocked status regression %s -> %s for task %s",
existing.get("status"), status, task_id)
return None
now = utc_now_iso()
return {
**existing,
**fields,
"task_id": task_id,
"status": status,
"ts": explicit_ts or str(existing.get("ts") or now),
"updated_at": now,
}
now = utc_now_iso()
ts = str(fields.pop("ts", "") or existing.get("ts") or now)
payload = {
**existing,
**fields,
"task_id": task_id,
"status": status,
"ts": ts,
"updated_at": now,
}
atomic_write_json(path, payload)
return payload
try:
return update_json_locked(path, _merge)
except TimeoutError:
# Last-resort visibility: a wedged sibling holding the lock must not
# silently drop a (possibly terminal) result. Log loudly and fall back
# to the previous unlocked merge so the durable record still lands.
log.error("task_results lock timeout for %s; falling back to unlocked merge", task_id)
existing = load_task_result(results_drive_root, task_id) or {}
merged = _merge(dict(existing))
if merged is None:
return existing
atomic_write_json(path, merged)
return merged

8
ouroboros/task_status.py
View file

@ -54,6 +54,14 @@ NONTERMINAL_STATUSES: frozenset[str] = frozenset({
STATUS_SCHEDULED,
STATUS_RUNNING,
})
# Truly settled outcomes (no cancel-intent latch): callers that must NOT treat
# cancel_requested as terminal (e.g. wait loops surfacing the final record).
SETTLED_STATUSES: frozenset[str] = frozenset({
STATUS_COMPLETED,
STATUS_FAILED,
STATUS_CANCELLED,
STATUS_REJECTED_DUPLICATE,
})
ARTIFACT_TERMINAL_STATUSES: frozenset[str] = frozenset({
ARTIFACT_STATUS_READY,
ARTIFACT_STATUS_FAILED,

109
ouroboros/tools/browser.py
View file

@ -22,9 +22,7 @@ except ImportError:
_HAS_STEALTH = False
from ouroboros.tools.registry import ToolContext, ToolEntry
from ouroboros.contracts.task_constraint import normalize_task_constraint
from ouroboros.server_auth import is_loopback_host
from ouroboros.tool_capabilities import LOCAL_READONLY_SUBAGENT_MODE
from ouroboros.config import AGENT_SERVER_PORT
log = logging.getLogger(__name__)
@ -141,6 +139,54 @@ def _is_blocked_subagent_ip(ip: ipaddress._BaseAddress) -> bool:
)
# AWS IMDSv6 endpoint; the IPv4 metadata services all live in 169.254.0.0/16
# (link-local), which ``is_link_local`` covers including decimal/hex URL
# spellings once ipaddress normalizes the resolved address.
_METADATA_IPV6_ADDRESSES = frozenset({ipaddress.ip_address("fd00:ec2::254")})
def _is_metadata_ip(ip: ipaddress._BaseAddress) -> bool:
# Unwrap IPv4-mapped IPv6 (http://[::ffff:169.254.169.254]/) so the
# link-local check sees the real IPv4 — mirrors mcp_client's guard.
mapped = getattr(ip, "ipv4_mapped", None)
if mapped is not None:
ip = mapped
return bool(ip.is_link_local) or ip in _METADATA_IPV6_ADDRESSES
def _is_metadata_blocked_browser_url(url: str) -> bool:
"""Main-agent guard: True only for link-local/cloud-metadata destinations."""
parsed = urlparse(str(url or ""))
if parsed.scheme not in {"http", "https"}:
return False
host = (parsed.hostname or "").strip().rstrip(".").lower()
if not host:
return False
try:
return _is_metadata_ip(ipaddress.ip_address(host))
except ValueError:
pass
if _NONSTANDARD_NUMERIC_IPV4_RE.match(host):
# Decimal/hex IPv4 spellings (e.g. http://2852039166/) bypass naive
# string checks; resolve via inet_aton normalization below.
try:
packed = socket.inet_aton(host)
return _is_metadata_ip(ipaddress.ip_address(socket.inet_ntoa(packed)))
except OSError:
return True
try:
infos = socket.getaddrinfo(host, None, type=socket.SOCK_STREAM)
except OSError:
return False # unresolvable hosts fail naturally at fetch time
for info in infos:
try:
if _is_metadata_ip(ipaddress.ip_address(str(info[4][0]))):
return True
except ValueError:
continue
return False
def _hostname_resolves_to_blocked_ip(host: str) -> bool:
try:
infos = socket.getaddrinfo(host, None, type=socket.SOCK_STREAM)
@ -505,6 +551,18 @@ def _ensure_browser(ctx: ToolContext, *, engine: str = "chromium", device: str =
if _is_subagent_blocked_browser_url(route.request.url, ctx)
else route.continue_(),
)
else:
# Main-agent SSRF guard (conservative): block ONLY link-local /
# cloud-metadata endpoints (169.254.0.0/16 incl. decimal/hex spellings,
# fd00:ec2::254). Private/LAN stays reachable — owners legitimately
# browse their own LAN services. Route interception re-validates every
# hop, so redirects cannot smuggle a metadata fetch.
bs_context.route(
"**/*",
lambda route: route.abort()
if _is_metadata_blocked_browser_url(route.request.url)
else route.continue_(),
)
return bs.page
@ -665,6 +723,49 @@ _MARKDOWN_JS = """() => {
}"""
def _inject_native_screenshot(ctx: ToolContext, b64: str) -> str:
"""Hand a fresh screenshot to a vision-capable active model natively.
The screenshot is saved to ``data/uploads/screenshots/<ts>.png`` (the
re-view path used by eviction placeholders) and injected as a user-role
image block via the existing multipart-preserving merge. The TOOL result
stays a plain string — the tool-message contract is unchanged. Non-vision
models keep the analyze_screenshot/vlm_query flow.
"""
try:
from ouroboros.provider_models import supports_vision
active_model = str(os.environ.get("OUROBOROS_MODEL", "") or "")
if not supports_vision(active_model):
return ""
messages = getattr(ctx, "messages", None)
if not isinstance(messages, list):
return ""
from ouroboros.utils import utc_now_iso
ts = utc_now_iso().replace(":", "").replace("-", "")[:15]
shot_dir = pathlib.Path(ctx.drive_root) / "uploads" / "screenshots"
shot_dir.mkdir(parents=True, exist_ok=True)
shot_path = shot_dir / f"{ts}.png"
shot_path.write_bytes(base64.b64decode(b64))
caption = f"[browser screenshot {ts}]"
from ouroboros.loop import _append_or_merge_user_content
_append_or_merge_user_content(messages, [
{"type": "text", "text": caption},
{
"type": "image_url",
"image_url": {"url": f"data:image/png;base64,{b64}"},
"_caption": caption,
"_source_path": str(shot_path),
},
])
return "The screenshot is attached to your context natively (vision model). "
except Exception:
log.debug("native screenshot injection failed", exc_info=True)
return ""
def _extract_page_output(page: Any, output: str, ctx: ToolContext) -> str:
"""Extract page content in the requested format."""
if output == "screenshot":
@ -676,9 +777,11 @@ def _extract_page_output(page: Any, output: str, ctx: ToolContext) -> str:
f"Screenshot captured ({len(b64)} bytes base64). "
"Use analyze_screenshot to inspect it."
)
native_note = _inject_native_screenshot(ctx, b64)
return (
f"Screenshot captured ({len(b64)} bytes base64). "
f"Call send_photo(image_base64='__last_screenshot__') to deliver it to the user."
+ (native_note or "Use analyze_screenshot to inspect it. ")
+ "Call send_photo(image_base64='__last_screenshot__') to deliver it to the user."
)
elif output == "html":
html = page.content()

2
ouroboros/tools/ci.py
View file

@ -10,7 +10,7 @@ import time
import urllib.error
import urllib.parse
import urllib.request
from typing import Any, Dict, List, Optional, Tuple
from typing import List, Optional, Tuple
from ouroboros.config import load_settings
from ouroboros.tools.registry import ToolContext, ToolEntry

34
ouroboros/tools/claude_advisory_review.py
View file

@ -21,10 +21,8 @@ from ouroboros.review_state import (
AdvisoryRunRecord,
AdvisoryReviewState,
compute_snapshot_hash,
format_status_section,
load_state,
make_repo_key,
save_state,
update_state,
_utc_now,
)
@ -926,6 +924,19 @@ def _record_bypass(ctx: ToolContext, state: "AdvisoryReviewState", snapshot_hash
))
update_state(drive_root, _mutate)
# Persistent visibility (same mechanism as advisory-enforcement overrides):
# review_status surfaces how often the advisory layer was bypassed/absent.
try:
from ouroboros.utils import update_json_locked, utc_now_iso as _now_iso
def _bump(current: dict) -> dict:
recent = list(current.get("recent") or [])
recent.append({"ts": _now_iso(), "block_reason": f"advisory_bypass: {reason}"[:200], "message_head": str(commit_message or "")[:200]})
return {"count": int(current.get("count") or 0) + 1, "recent": recent[-10:]}
update_json_locked(pathlib.Path(drive_root) / "state" / "advisory_overrides.json", _bump)
except Exception:
log.debug("Failed to persist advisory bypass visibility", exc_info=True)
if "ANTHROPIC_API_KEY" in reason:
msg = (
"⚠️ ANTHROPIC_API_KEY is not set — advisory review skipped automatically. "
@ -1406,11 +1417,20 @@ def _handle_advisory_pre_review(
prompt_chars=prompt_chars, model_used=model_used,
session_id=advisory_session_id, duration_sec=_advisory_duration,
)
state.add_run(run)
# Locked read-modify-write against the LIVE ledger: the SDK call above runs
# for minutes, and a state object loaded before it would clobber stale-marks
# and concurrent runs recorded meanwhile (the pre-SDK `state` snapshot is
# only used for gating decisions, never persisted from here on).
def _record_run(live_state: "AdvisoryReviewState") -> None:
live_state.add_run(run)
if run_status != "parse_failure" and items:
_resolve_matching_obligations(live_state, items, snapshot_hash, repo_key=repo_key)
update_state(drive_root, _record_run)
# Surface parse failures explicitly.
if run_status == "parse_failure":
save_state(drive_root, state)
return _json_response({
"status": "parse_failure",
"snapshot_hash": snapshot_hash,
@ -1424,12 +1444,6 @@ def _handle_advisory_pre_review(
),
})
# Resolve only unambiguous PASSed obligations, even if unrelated findings fail.
if items:
_resolve_matching_obligations(state, items, snapshot_hash, repo_key=repo_key)
save_state(drive_root, state)
# Build human-readable summary.
findings_summary: List[str] = []
for item in critical_fails:

2
ouroboros/tools/compact_context.py
View file

@ -3,7 +3,7 @@
from __future__ import annotations
import logging
from typing import Any, Dict, List
from typing import List
from ouroboros.tools.registry import ToolEntry

3
ouroboros/tools/core.py
View file

@ -26,7 +26,6 @@ from ouroboros.tool_access import (
resource_root_path,
user_files_path_block_reason,
)
from ouroboros.tool_capabilities import LOCAL_READONLY_SUBAGENT_MODE
from ouroboros.utils import atomic_write_json, read_text, safe_relpath, utc_now_iso
from ouroboros.contracts.task_constraint import normalize_task_constraint, resolve_payload_path
from ouroboros.contracts.skill_payload_policy import (
@ -1428,7 +1427,7 @@ def _codebase_digest(ctx: ToolContext) -> str:
def _forward_to_worker(ctx: ToolContext, task_id: str, message: str) -> str:
"""Forward a message to a running worker task's mailbox."""
from ouroboros.owner_inject import write_owner_message
from ouroboros.owner_mailbox import write_owner_message
from ouroboros.task_results import STATUS_RUNNING, validate_task_id
from ouroboros.task_status import FINAL_STATUSES, load_effective_task_result

11
ouroboros/tools/extension_dispatch.py
View file

@ -57,9 +57,14 @@ def dispatch_extension_tool(ctx: Any, name: str, ext_tool: Dict[str, Any], args:
handler = ext_tool["handler"]
try:
result = handler(ctx, **call_args)
except TypeError:
result = handler(**call_args)
# Signature-based dispatch (single execution): the old TypeError retry
# re-ran handlers whose BODY raised TypeError after side effects.
from ouroboros.extension_process_runner import _handler_wants_ctx
if _handler_wants_ctx(handler):
result = handler(ctx, **call_args)
else:
result = handler(**call_args)
except Exception as exc:
return f"⚠️ TOOL_ERROR ({name}): extension tool failed: {type(exc).__name__}: {exc}"

7
ouroboros/tools/git.py
View file

@ -9,7 +9,6 @@ import os
import pathlib
import re
import subprocess
import sys
import time
from typing import Any, Dict, List, Optional
@ -1563,7 +1562,7 @@ def get_tools() -> List[ToolEntry]:
"name": "vcs_pull_ff",
"description": "Fetch from origin and fast-forward merge. Safe: never rewrites history.",
"parameters": {"type": "object", "properties": {}, "required": []},
}, _pull_from_remote, is_code_tool=True),
}, _pull_from_remote, is_code_tool=True, mutates_worktree=True),
ToolEntry("vcs_restore", {
"name": "vcs_restore",
"description": "Discard uncommitted changes, restoring to last committed state (HEAD).",
@ -1571,7 +1570,7 @@ def get_tools() -> List[ToolEntry]:
"confirm": {"type": "boolean", "description": "Must be true to execute."},
"paths": {"type": "array", "items": {"type": "string"}, "description": "Specific files to restore"},
}, "required": ["confirm"]},
}, _restore_to_head, is_code_tool=True),
}, _restore_to_head, is_code_tool=True, mutates_worktree=True),
ToolEntry("vcs_revert", {
"name": "vcs_revert",
"description": "Revert a specific commit by creating a new undo commit. Safe: no history rewrite.",
@ -1579,5 +1578,5 @@ def get_tools() -> List[ToolEntry]:
"sha": {"type": "string", "description": "Commit SHA to revert"},
"confirm": {"type": "boolean", "description": "Must be true to execute."},
}, "required": ["sha", "confirm"]},
}, _revert_commit, is_code_tool=True),
}, _revert_commit, is_code_tool=True, mutates_worktree=True),
]

38
ouroboros/tools/git_pr.py
View file

@ -10,7 +10,6 @@ import subprocess
from typing import List, Optional
from ouroboros.tools.registry import ToolContext, ToolEntry
from ouroboros.tools.commit_gate import _invalidate_advisory
from ouroboros.tools.git import _acquire_git_lock, _release_git_lock, _sanitize_git_error
log = logging.getLogger(__name__)
@ -216,11 +215,6 @@ def _rollback_failed_amend(
amend_err = (amend_r.stderr or amend_r.stdout or "").strip()
_g(["reset", "--hard", "HEAD~1"], repo_dir)
applied.pop()
if applied:
_invalidate_advisory(
ctx, changed_paths=[], mutation_root=repo_dir,
source_tool="cherry_pick_pr_commits",
)
return (
f"⚠️ CHERRY_PICK_ERROR: author amend failed on {sha[:12]} "
f"(rolled back to pre-commit state):\n{amend_err[:500]}\n\n"
@ -327,11 +321,6 @@ def _cherry_pick_pr_commits(
if override_author is not None:
date_r = _g(["log", "-1", "--format=%aI", sha], repo_dir)
if date_r.returncode != 0 or not date_r.stdout.strip():
if applied:
_invalidate_advisory(
ctx, changed_paths=[], mutation_root=repo_dir,
source_tool="cherry_pick_pr_commits",
)
return (
f"⚠️ CHERRY_PICK_ERROR: Cannot read author date for {sha[:12]} "
f"(git log returned {date_r.returncode}). Aborting before "
@ -346,11 +335,6 @@ def _cherry_pick_pr_commits(
err = (result.stderr or result.stdout or "").strip()
_g(["cherry-pick", "--abort"], repo_dir)
if stop_on_conflict:
if applied:
_invalidate_advisory(
ctx, changed_paths=[], mutation_root=repo_dir,
source_tool="cherry_pick_pr_commits",
)
return (
f"⚠️ CHERRY_PICK_CONFLICT on {sha[:12]}:\n{err[:500]}\n\n"
f"Applied before conflict: {[s[:12] for s in applied] or 'none'}\n"
@ -384,9 +368,6 @@ def _cherry_pick_pr_commits(
f"Skipped (conflict): {skipped}"
)
_invalidate_advisory(ctx, changed_paths=[], mutation_root=repo_dir,
source_tool="cherry_pick_pr_commits")
override_note = ""
if override_author is not None:
override_note = (
@ -460,9 +441,6 @@ def _stage_adaptations(ctx: ToolContext) -> str:
finally:
_release_git_lock(lock)
_invalidate_advisory(ctx, changed_paths=[], mutation_root=repo_dir,
source_tool="stage_adaptations")
files = staged.splitlines()
return (
f"✅ Staged {len(files)} file(s) on '{current_branch}' (NOT committed):\n"
@ -519,7 +497,6 @@ def _stage_pr_merge(
)
adaptation_patch: bytes = b""
adapt_paths: list = []
staged_before = _g(["diff", "--cached", "--name-only"], repo_dir).stdout.strip()
if staged_before:
diff_r = subprocess.run(
@ -533,7 +510,7 @@ def _stage_pr_merge(
f"Staged changes are preserved. Fix the repo state and retry."
)
adaptation_patch = diff_r.stdout
adapt_paths = staged_before.splitlines()
staged_before.splitlines()
_g(["reset", "--hard", "HEAD"], repo_dir)
def _restore_on_error() -> None:
@ -609,9 +586,6 @@ def _stage_pr_merge(
if log_r.returncode == 0:
authors = sorted(set(log_r.stdout.strip().splitlines()))
_invalidate_advisory(ctx, changed_paths=[], mutation_root=repo_dir,
source_tool="stage_pr_merge")
co_authored = "\n".join(f"Co-authored-by: {a}" for a in authors) if authors else ""
author_hint = (
f"\n\nAttribution — include in commit_reviewed message:\n{co_authored}"
@ -645,7 +619,7 @@ def get_tools() -> List[ToolEntry]:
"remote": {"type": "string", "default": "origin",
"description": "Git remote name (default: origin)"},
}, "required": ["pr_number"]},
}, _fetch_pr_ref, is_code_tool=True),
}, _fetch_pr_ref, is_code_tool=True, mutates_worktree=True),
ToolEntry("create_integration_branch", {
"name": "create_integration_branch",
@ -659,7 +633,7 @@ def get_tools() -> List[ToolEntry]:
"base_branch": {"type": "string", "default": "ouroboros",
"description": "Branch to create from"},
}, "required": ["pr_number"]},
}, _create_integration_branch, is_code_tool=True),
}, _create_integration_branch, is_code_tool=True, mutates_worktree=True),
ToolEntry("cherry_pick_pr_commits", {
"name": "cherry_pick_pr_commits",
@ -703,7 +677,7 @@ def get_tools() -> List[ToolEntry]:
"additionalProperties": False,
},
}, "required": ["shas"]},
}, _cherry_pick_pr_commits, is_code_tool=True),
}, _cherry_pick_pr_commits, is_code_tool=True, mutates_worktree=True),
ToolEntry("stage_adaptations", {
"name": "stage_adaptations",
@ -715,7 +689,7 @@ def get_tools() -> List[ToolEntry]:
"Must be on an integrate/pr-N branch."
),
"parameters": {"type": "object", "properties": {}},
}, _stage_adaptations, is_code_tool=True),
}, _stage_adaptations, is_code_tool=True, mutates_worktree=True),
ToolEntry("stage_pr_merge", {
"name": "stage_pr_merge",
@ -732,5 +706,5 @@ def get_tools() -> List[ToolEntry]:
"branch": {"type": "string",
"description": "Integration branch to merge (e.g. integrate/pr-17)"},
}, "required": ["branch"]},
}, _stage_pr_merge, is_code_tool=True),
}, _stage_pr_merge, is_code_tool=True, mutates_worktree=True),
]

2
ouroboros/tools/github.py
View file

@ -278,7 +278,7 @@ def _get_pr(ctx: ToolContext, number: int) -> str:
diff_raw = _gh_cmd(["pr", "diff", str(number)], ctx, timeout=60)
if not diff_raw.startswith("⚠️") and diff_raw.strip():
lines.append(f"\n**Diff (truncated to 8000 chars):**\n```diff")
lines.append("\n**Diff (truncated to 8000 chars):**\n```diff")
lines.append(_truncate_with_notice(diff_raw, 8000))
lines.append("```")

1
ouroboros/tools/health.py
View file

@ -3,7 +3,6 @@
import logging
import os
import pathlib
from typing import Any, Dict
from ouroboros.tools.registry import ToolContext, ToolEntry

22
ouroboros/tools/knowledge.py
View file

@ -200,7 +200,27 @@ def _knowledge_write(ctx: ToolContext, topic: str, content: str, mode: str = "ov
_ensure_dir(ctx)
old_content = path.read_text(encoding="utf-8") if path.exists() else ""
if mode == "append":
if sanitized_topic == "improvement-backlog":
# The backlog is concurrently rewritten by its locked helpers (groomer,
# post-task promotion close). A generic unlocked write can interleave
# with a groom and lose items — take the SAME file lock for the write.
from ouroboros.improvement_backlog import _locked_text_file
# "a+" for both modes: "r+" raises FileNotFoundError on the very first
# overwrite of a fresh drive (the topic file is created on first write).
path.parent.mkdir(parents=True, exist_ok=True)
with _locked_text_file(path, mode="a+") as fh:
if mode == "append":
fh.seek(0)
tail = fh.read()
if tail and not tail.endswith("\n"):
fh.write("\n")
fh.write(content)
else:
fh.seek(0)
fh.truncate(0)
fh.write(content)
elif mode == "append":
needs_newline = False
if path.exists() and path.stat().st_size > 0:
with open(path, "rb") as rf:

1
ouroboros/tools/memory_tools.py
View file

@ -6,7 +6,6 @@ from pathlib import Path
from typing import List
from ouroboros.tools.registry import ToolEntry, ToolContext
from ouroboros.utils import utc_now_iso
log = logging.getLogger(__name__)

2
ouroboros/tools/parallel_review.py
View file

@ -4,12 +4,10 @@ from __future__ import annotations
import concurrent.futures as _cf
import hashlib
import logging
from typing import Optional
from ouroboros.utils import run_cmd
from ouroboros.tools.review_helpers import build_scope_actor_record, format_review_history_entry
from ouroboros.tools.scope_review import (
ScopeReviewResult,
run_scope_review,
_degraded_scope_requested,
_get_scope_model,

12
ouroboros/tools/plan_review.py
View file

@ -63,7 +63,15 @@ def _effective_swarm_max_wait() -> float:
from ouroboros.tools.review_helpers import REVIEW_PROMPT_TOKEN_BUDGET as _REVIEW_BUDGET
_PLAN_BUDGET_TOKEN_LIMIT = _REVIEW_BUDGET
# Reserve output headroom inside the reviewer's 1M window (same class of fix as
# scope_review/deep_self_review): SSOT input budget + max output must not
# exceed the window, or atlas-heavy plan packs hit a deterministic provider 400.
_PLAN_MODEL_CONTEXT_WINDOW = 1_000_000
_PLAN_OUTPUT_MARGIN_TOKENS = 155_000
_PLAN_BUDGET_TOKEN_LIMIT = min(
_REVIEW_BUDGET,
_PLAN_MODEL_CONTEXT_WINDOW - _PLAN_REVIEW_MAX_TOKENS - _PLAN_OUTPUT_MARGIN_TOKENS,
)
def get_tools():
@ -711,7 +719,7 @@ async def _run_plan_review_async(
if atlas.status == "budget_exceeded":
estimated = int((atlas.manifest or {}).get("estimated_total_tokens") or 0)
return (
f"⚠️ PLAN_REVIEW_SKIPPED: generated repository atlas exceeded hard budget"
"⚠️ PLAN_REVIEW_SKIPPED: generated repository atlas exceeded hard budget"
+ (f" ({estimated:,} estimated tokens)" if estimated else "")
+ ". Split the plan into a smaller scope or choose a smaller context_level."
)

1
ouroboros/tools/query_code.py
View file

@ -2,7 +2,6 @@
from __future__ import annotations
import json
import pathlib
import re
from typing import Any, List

57
ouroboros/tools/registry.py
View file

@ -4,7 +4,6 @@ from __future__ import annotations
import copy
import hashlib
import json
import logging
import os
import pathlib
@ -15,7 +14,6 @@ from typing import Any, Callable, Dict, List, Optional
from ouroboros.runtime_mode_policy import (
PROTECTED_RUNTIME_PATHS,
core_patch_notice,
mode_allows_protected_write,
protected_paths_in,
protected_write_block_message,
@ -55,7 +53,7 @@ from ouroboros.protected_artifacts import shell_block_reason as protected_artifa
from ouroboros.git_shell_policy import run_shell_git_block_reason, workspace_git_safety_violation
from ouroboros.tool_access import light_cognitive_or_root_redirect, normalize_root, resolve_shell_cwd, workspace_mode_block_reason
from ouroboros.utils import safe_relpath
from ouroboros.contracts.task_constraint import TaskConstraint, VALID_WRITE_SURFACES, normalize_task_constraint, resolve_payload_path
from ouroboros.contracts.task_constraint import TaskConstraint, VALID_WRITE_SURFACES, normalize_task_constraint
from ouroboros.contracts.skill_payload_policy import (
SKILL_OWNER_STATE_FILENAMES,
SKILL_OWNER_STATE_STEMS,
@ -599,6 +597,11 @@ class ToolEntry:
handler: Callable # fn(ctx: ToolContext, **args) -> str
is_code_tool: bool = False
timeout_sec: int = 360
# Capability flag: tool can mutate the live repo worktree. The dispatcher
# snapshots `git status --porcelain` around flagged tools and invalidates
# advisory freshness when the worktree ACTUALLY changed — covering error
# and timeout paths uniformly, and never invalidating for read-only runs.
mutates_worktree: bool = False
class ToolRegistry:
@ -1731,12 +1734,23 @@ class ToolRegistry:
if name in _PROCESS_COMMAND_TOOLS and workspace_mode
else None
)
worktree_before = (
self._worktree_status_snapshot() if entry.mutates_worktree else None
)
try:
result = entry.handler(self._ctx, **args)
except TypeError as e:
return f"⚠️ TOOL_ARG_ERROR ({name}): {e}"
except Exception as e:
return f"⚠️ TOOL_ERROR ({name}): {e}"
try:
result = entry.handler(self._ctx, **args)
except TypeError as e:
return f"⚠️ TOOL_ARG_ERROR ({name}): {e}"
except Exception as e:
return f"⚠️ TOOL_ERROR ({name}): {e}"
finally:
# Central advisory invalidation by OBSERVED worktree diff: runs on
# success, tool error, and exception paths alike (the per-tool
# manual calls missed early-return/error paths), and skips
# invalidation when a flagged tool ran read-only.
if worktree_before is not None:
self._invalidate_advisory_if_worktree_changed(name, worktree_before)
if name in _PROCESS_COMMAND_TOOLS:
result = self._run_shell_post_checks(
result,
@ -1750,6 +1764,31 @@ class ToolRegistry:
return f"{safety_msg}\n\n---\n{result}"
return result
def _worktree_status_snapshot(self) -> str:
try:
from ouroboros.utils import run_cmd
return run_cmd(["git", "status", "--porcelain"], cwd=self._ctx.repo_dir, timeout=20)
except Exception:
return "<status-unavailable>"
def _invalidate_advisory_if_worktree_changed(self, tool_name: str, before: str) -> None:
after = self._worktree_status_snapshot()
if after == before:
return
try:
from ouroboros.review_state import invalidate_advisory_after_mutation
invalidate_advisory_after_mutation(
pathlib.Path(self._ctx.drive_root),
mutation_root=pathlib.Path(self._ctx.repo_dir),
source_tool=tool_name,
)
except Exception:
logging.getLogger(__name__).debug(
"Central advisory invalidation failed for %s", tool_name, exc_info=True
)
def override_handler(self, name: str, handler) -> None:
"""Override the handler for a registered tool (used for closure injection)."""
entry = self._entries.get(name)
@ -1758,7 +1797,9 @@ class ToolRegistry:
name=entry.name,
schema=entry.schema,
handler=handler,
is_code_tool=entry.is_code_tool,
timeout_sec=entry.timeout_sec,
mutates_worktree=entry.mutates_worktree,
)
@property

54
ouroboros/tools/review.py
View file

@ -12,6 +12,7 @@ from ouroboros.utils import (
run_cmd,
append_jsonl,
truncate_review_artifact,
utc_now_iso,
)
from ouroboros import config as _cfg
from ouroboros.tools.registry import ToolEntry, ToolContext
@ -90,7 +91,6 @@ from ouroboros.tools.review_helpers import (
REPO_ANTI_PATTERN_LOCK_GUARD,
REVIEW_JSON_ARRAY_CONTRACT,
REVIEW_PREAMBLE,
normalize_reviewer_items,
build_self_verification_template,
build_review_history_section as _build_review_history_section,
emit_review_usage,
@ -758,12 +758,54 @@ def _handle_review_block_or_warning(
"""Either block immediately or downgrade to advisory warning."""
if blocking_review:
return blocked_msg
_record_advisory_override(ctx, blocked_msg)
_append_review_warning(ctx, advisory_prefix + blocked_msg)
ctx._review_iteration_count = 0
ctx._review_history = []
return None
def _record_advisory_override(ctx: ToolContext, blocked_msg: str) -> None:
"""Durable trace of a blocking signal waved through by advisory enforcement.
Constitutional requirement (BIBLE P3 "Owner-chosen enforcement, loud
advisory"): every decision blocking enforcement would have stopped must
leave a durable, owner-visible trace. Persisted to events.jsonl AND to a
persistent counter file surfaced by the review_status tool.
"""
reason = str(getattr(ctx, "_last_review_block_reason", "") or "unknown")
try:
append_jsonl(ctx.drive_logs() / "events.jsonl", {
"ts": utc_now_iso(),
"type": "review_advisory_override",
"block_reason": reason,
"message_head": str(blocked_msg or "")[:600],
"task_id": str(getattr(ctx, "task_id", "") or ""),
})
except Exception:
log.debug("Failed to emit review_advisory_override event", exc_info=True)
try:
from ouroboros.utils import update_json_locked
path = ctx.drive_root / "state" / "advisory_overrides.json"
def _bump(current: dict) -> dict:
recent = list(current.get("recent") or [])
recent.append({
"ts": utc_now_iso(),
"block_reason": reason,
"message_head": str(blocked_msg or "")[:300],
})
return {
"count": int(current.get("count") or 0) + 1,
"recent": recent[-10:],
}
update_json_locked(path, _bump)
except Exception:
log.warning("Failed to persist advisory override visibility", exc_info=True)
def _collect_review_findings(ctx: ToolContext, model_results: list) -> tuple[list[str], list[str], list[str], list[dict]]:
parsed = parse_model_review_results({"results": model_results})
critical_fails: List[str] = []
@ -1094,6 +1136,7 @@ def _run_unified_review(ctx: ToolContext, commit_message: str,
ctx, commit_message, critical_fails, advisory_warns, errored_note,
)
_record_advisory_override(ctx, "; ".join(critical_fails[:5]))
_append_review_warning(
ctx,
"Review enforcement=Advisory: critical review findings did not block commit.",
@ -1105,9 +1148,12 @@ def _run_unified_review(ctx: ToolContext, commit_message: str,
if errored_note:
_append_review_warning(ctx, errored_note)
# All clear: reset iteration state.
ctx._review_iteration_count = 0
ctx._review_history = []
if not critical_fails:
# All clear: reset iteration state. With critical findings present
# (advisory enforcement), the anti-thrashing history must SURVIVE so
# repeat findings on the next attempt are still recognized as repeats.
ctx._review_iteration_count = 0
ctx._review_history = []
if errored_note:
advisory_warns.append(errored_note.strip())

4
ouroboros/tools/review_helpers.py
View file

@ -282,6 +282,9 @@ Return ONLY a JSON array. Each element:
"severity": "critical" | "advisory",
"reason": "<for FAIL: file, line/symbol, what is wrong, how to fix>"
}
If you reviewed everything and found NOTHING to report, return the empty array
followed by the sentinel word NO_FINDINGS on its own line. An empty array
without NO_FINDINGS is treated as a non-response and excluded from quorum.
"""
REVIEW_SEVERITY_THRESHOLDS = """\
@ -1231,7 +1234,6 @@ def build_scope_section(scope: str = "") -> str:
def get_advisory_runtime_diagnostics(model: str, prompt_chars: int,
touched_paths: list) -> dict:
"""Collect best-effort advisory SDK diagnostics; never raises."""
import sys
diag: dict = {
"model": model,

13
ouroboros/tools/review_synthesis.py
View file

@ -131,7 +131,10 @@ def _parse_synthesis_output(raw: str) -> Optional[List[Dict[str, Any]]]:
continue
canonical = {
"item": str(entry.get("item", "") or ""),
"severity": str(entry.get("severity", "advisory") or "advisory"),
# The synthesizer's INPUT is exclusively critical findings, so a
# missing severity must stay critical — an "advisory" default
# silently downgraded blocking findings out of the gate.
"severity": str(entry.get("severity", "critical") or "critical"),
"reason": str(entry.get("reason", "") or ""),
"obligation_id": str(entry.get("obligation_id", "") or ""),
"evidence_from_reviewers": _normalize_evidence(entry.get("evidence_from_reviewers")),
@ -183,16 +186,16 @@ def synthesize_to_canonical_issues(
try:
raw_response = _call_synthesis_llm(prompt, ctx=ctx)
except Exception as exc:
log.debug("review_synthesis: LLM call raised exception: %s — using original findings", exc)
log.warning("review_synthesis: LLM call raised exception: %s — using original findings", exc)
return critical_findings
if raw_response is None:
log.debug("review_synthesis: LLM call returned None — using original findings")
log.warning("review_synthesis: LLM call returned None — using original findings")
return critical_findings
canonical = _parse_synthesis_output(raw_response)
if canonical is None:
log.debug("review_synthesis: failed to parse LLM output — using original findings")
log.warning("review_synthesis: failed to parse LLM output — using original findings")
return critical_findings
log.debug(
@ -249,7 +252,7 @@ def _call_synthesis_llm(prompt: str, *, ctx: Any = None) -> Optional[str]:
return str(content) if content else None
except Exception as exc:
log.debug("review_synthesis: LLM call failed: %s", exc)
log.warning("review_synthesis: LLM call failed: %s", exc)
return None

16
ouroboros/tools/scope_review.py
View file

@ -9,7 +9,6 @@ fail closed; oversized prompts are the explicit non-blocking skip path.
from __future__ import annotations
import asyncio
import contextvars
import inspect
import logging
@ -119,7 +118,7 @@ class _ScopeAtlasBudgetExceeded(RuntimeError):
self.manifest = dict(manifest or {})
token_count = int(self.manifest.get("estimated_total_tokens") or 0)
super().__init__(
f"Generated Scope Atlas exceeded hard budget"
"Generated Scope Atlas exceeded hard budget"
+ (f" (~{token_count:,} estimated tokens)" if token_count else "")
)
@ -410,10 +409,15 @@ def _build_scope_prompt(
) -> tuple:
"""Build the scope prompt or a touched-context/budget status sentinel."""
_SCOPE_CONTEXT_MANIFEST.set({})
try:
scope_checklist = load_checklist_section("Intent / Scope Review Checklist")
except Exception:
scope_checklist = "(Intent / Scope Review Checklist not found in docs/CHECKLISTS.md)"
# Fail-closed (immune-system parity with the triad): a scope review running
# WITHOUT its checklist silently reviews against nothing. Raising turns it
# into an explicit SCOPE_REVIEW_BLOCKED error instead of a placeholder pass.
scope_checklist = load_checklist_section("Intent / Scope Review Checklist")
if not str(scope_checklist or "").strip():
raise RuntimeError(
"Intent / Scope Review Checklist could not be loaded from docs/CHECKLISTS.md — "
"scope review cannot run without its checklist (fail-closed)."
)
goal_section = build_goal_section(goal, scope, commit_message)
scope_section = build_scope_section(scope)

17
ouroboros/tools/services.py
View file

@ -19,7 +19,6 @@ from ouroboros.platform_layer import (
kill_process_group_id,
kill_process_tree,
process_group_id,
subprocess_new_group_kwargs,
)
from ouroboros.tools.registry import ToolContext, ToolEntry
from ouroboros.tool_access import resolve_shell_cwd
@ -378,14 +377,22 @@ def _start_service(
log_fh = log_path.open("ab")
try:
bootstrap_process_path()
proc = subprocess.Popen(
# Supervised spawn: the durable custody record means a SIGKILLed worker
# can no longer orphan this service invisibly — the reaper finds it in
# the ledger on the next server generation.
from ouroboros.process_custody import spawn_supervised
proc = spawn_supervised(
[str(part) for part in cmd],
drive_root=pathlib.Path(ctx.drive_root),
purpose=f"service:{service_name}",
scope="task",
owner_task_id=task_id,
cwd=str(workdir),
stdout=log_fh,
stderr=subprocess.STDOUT,
stdin=subprocess.DEVNULL,
env=_service_env(),
**subprocess_new_group_kwargs(),
)
pgid = process_group_id(proc.pid)
log_fh.close()
@ -721,7 +728,7 @@ def get_tools() -> List[ToolEntry]:
"readiness": {"type": "object", "default": {}, "description": "Optional {log_contains|stdout_contains, timeout_sec} readiness probe."},
"outputs": {"type": "array", "items": {"type": "string"}, "default": [], "description": "Files generated by the service to copy into the task artifact store when the service stops."},
}, "required": ["cmd"]},
}, _start_service, is_code_tool=True, timeout_sec=30),
}, _start_service, is_code_tool=True, timeout_sec=30, mutates_worktree=True),
ToolEntry("service_status", {
"name": "service_status",
"description": "Return pid/state/readiness/uptime for a task-scoped service.",
@ -743,5 +750,5 @@ def get_tools() -> List[ToolEntry]:
"parameters": {"type": "object", "properties": {
"name": {"type": "string", "default": "service"},
}, "required": []},
}, _stop_service, is_code_tool=True, timeout_sec=30),
}, _stop_service, is_code_tool=True, timeout_sec=30, mutates_worktree=True),
]

16
ouroboros/tools/shell.py
View file

@ -22,7 +22,7 @@ from subprocess import Popen, CompletedProcess
from typing import Any, Dict, List
from ouroboros.artifacts import artifact_store_path_block_reason, copy_directory_to_task_artifacts, copy_file_to_task_artifacts
from ouroboros.platform_layer import IS_WINDOWS, bootstrap_process_path, kill_process_tree, subprocess_new_group_kwargs
from ouroboros.platform_layer import bootstrap_process_path, kill_process_tree, subprocess_new_group_kwargs
from ouroboros.config import get_runtime_mode, load_settings
from ouroboros.runtime_mode_policy import (
core_patch_notice,
@ -973,6 +973,9 @@ def _run_shell(ctx: ToolContext, cmd, cwd: str = "", outputs: List[str] | None =
return autocorrect_note + f"⚠️ SHELL_EXIT_ERROR: command exited with {_describe_returncode(res.returncode, cwd=work_dir)}.\n\n{_format_process_output(res.stdout or '', res.stderr or '')}{executor_note}"
after_changed = _status_snapshot(repo_root)
if after_changed != before_changed:
# Kept (nonstandard case): repo_root here is the RESOLVED cwd root,
# which may be a workspace/skill repo the central live-repo
# dispatcher check does not watch; this call passes precise paths.
_invalidate_advisory(
ctx,
changed_paths=after_changed or before_changed,
@ -1008,7 +1011,7 @@ def _run_shell(ctx: ToolContext, cmd, cwd: str = "", outputs: List[str] | None =
if artifact_failed:
return (
autocorrect_note
+ f"⚠️ ARTIFACT_OUTPUT_ERROR: command succeeded but declared output registration failed. "
+ "⚠️ ARTIFACT_OUTPUT_ERROR: command succeeded but declared output registration failed. "
+ f"{_describe_returncode(0, cwd=work_dir)}\n"
+ f"{_format_process_output(res.stdout or '', res.stderr or '')}"
+ artifact_note
@ -1344,6 +1347,9 @@ def _claude_code_edit(ctx: ToolContext, prompt: str, cwd: str = "", budget: floa
after_changed = _status_snapshot(target_repo_root)
if repo_mode and after_changed != before_changed:
# Kept (nonstandard case): target_repo_root may be a skill
# payload/workspace repo outside the central live-repo check,
# and result.changed_files gives precise invalidation paths.
_invalidate_advisory(
ctx,
changed_paths=result.changed_files or after_changed or before_changed,
@ -1534,7 +1540,7 @@ def get_tools() -> List[ToolEntry]:
"description": "Generated file paths to copy/register into the task artifact store after success.",
},
}, "required": ["cmd"]},
}, _run_shell, is_code_tool=True, timeout_sec=_RUN_SHELL_DEFAULT_TIMEOUT_SEC),
}, _run_shell, is_code_tool=True, timeout_sec=_RUN_SHELL_DEFAULT_TIMEOUT_SEC, mutates_worktree=True),
ToolEntry("claude_code_edit", {
"name": "claude_code_edit",
"description": (
@ -1568,7 +1574,7 @@ def get_tools() -> List[ToolEntry]:
"description": "Generated file paths to copy/register into the task artifact store after a successful edit.",
},
}, "required": ["prompt"]},
}, _claude_code_edit, is_code_tool=True, timeout_sec=1200),
}, _claude_code_edit, is_code_tool=True, timeout_sec=1200, mutates_worktree=True),
ToolEntry("run_script", {
"name": "run_script",
"description": (
@ -1588,5 +1594,5 @@ def get_tools() -> List[ToolEntry]:
"description": "Generated file paths to copy/register into the task artifact store after success.",
},
}, "required": ["script"]},
}, _run_script, is_code_tool=True, timeout_sec=_RUN_SHELL_DEFAULT_TIMEOUT_SEC),
}, _run_script, is_code_tool=True, timeout_sec=_RUN_SHELL_DEFAULT_TIMEOUT_SEC, mutates_worktree=True),
]

20
ouroboros/tools/skill_exec.py
View file

@ -679,6 +679,26 @@ def _handle_skill_exec(
except Exception:
log.debug("Could not augment skill env with isolated dependencies", exc_info=True)
# TOCTOU narrowing: re-hash the payload immediately before spawn. The
# gate-time hash above ran before grants/env/deps resolution — a write
# landing in that window would execute unreviewed code under a PASS verdict.
try:
spawn_hash = compute_content_hash(
loaded.skill_dir,
manifest_entry=loaded.manifest.entry,
manifest_scripts=loaded.manifest.scripts,
)
except SkillPayloadUnreadable as exc:
return (
f"⚠️ SKILL_EXEC_ERROR: skill {skill_name!r} payload became unreadable "
f"right before execution ({exc})."
)
if spawn_hash != current_hash:
return (
f"⚠️ SKILL_EXEC_BLOCKED: skill {skill_name!r} payload changed between "
"the review-freshness check and execution. Re-run skill_review."
)
try:
returncode, stdout_bytes, stderr_bytes, overflowed = _run_skill_subprocess(
cmd,

8
ouroboros/tools/tool_discovery.py
View file

@ -1,13 +1,13 @@
"""Tool discovery meta-tools.
v6.17 starts tasks with the full selected capability envelope. These handlers
Tasks start with the full selected capability envelope. These handlers
remain as a compatibility/discovery surface: they can confirm whether a named
tool is registered, but they no longer grant delayed core capabilities.
"""
from __future__ import annotations
import logging
from typing import Any, Dict, List, Optional, TYPE_CHECKING
from typing import List, Optional, TYPE_CHECKING
from ouroboros.tools.registry import ToolContext, ToolEntry
from ouroboros.tool_policy import list_non_core_tools as _policy_list_non_core
@ -91,7 +91,7 @@ def get_tools() -> List[ToolEntry]:
"name": "list_available_tools",
"description": (
"List tools omitted from the active envelope, if any. In the normal "
"v6.17 envelope this usually reports that all tools are already active."
"full-envelope model this usually reports that all tools are already active."
),
"parameters": {
"type": "object",
@ -106,7 +106,7 @@ def get_tools() -> List[ToolEntry]:
schema={
"name": "enable_tools",
"description": (
"Compatibility check for named tools (comma-separated). v6.17 starts "
"Compatibility check for named tools (comma-separated). Tasks start "
"with the selected envelope active, so this confirms registration instead "
"of granting delayed core tools."
),

5
ouroboros/tools/vision.py
View file

@ -3,6 +3,7 @@
from __future__ import annotations
import logging
import pathlib
import os
from typing import Any, Dict, List, Tuple
@ -189,8 +190,8 @@ def _vlm_query(ctx: ToolContext, prompt: str, image_url: str = "", image_base64:
mime = _detect_image_mime_for_vlm(raw)
if not mime:
return (
f"⚠️ File does not appear to be a supported image (PNG/JPEG/GIF/WEBP). "
f"Only image files may be sent to the VLM via file_path."
"⚠️ File does not appear to be a supported image (PNG/JPEG/GIF/WEBP). "
"Only image files may be sent to the VLM via file_path."
)
images.append(_image_payload_from_bytes(raw, mime))
elif image_url:

30
ouroboros/triad_review.py
View file

@ -165,6 +165,28 @@ def _normalize_items(items: List[Any]) -> List[Any]:
return items
def _empty_array_is_verified_clean(raw_text: str) -> bool:
"""True when an EMPTY findings array is a verifiable clean verdict.
Accepted markers: the explicit ``NO_FINDINGS`` sentinel anywhere in the
response, or a response whose entire body (modulo code fences) is the bare
array itself.
"""
text = str(raw_text or "")
if "NO_FINDINGS" in text:
return True
stripped = text.strip()
if "```" in stripped:
# Unwrap a single fenced block: ```json\n[]\n``` or ```\n[]\n```.
parts = [chunk.strip() for chunk in stripped.split("```") if chunk.strip()]
if len(parts) == 1:
body = parts[0]
if body.startswith("json"):
body = body[4:].strip()
stripped = body
return stripped == "[]"
def parse_model_review_results(
result_json: Dict[str, Any],
*,
@ -192,6 +214,14 @@ def parse_model_review_results(
if parsed is None:
records.append(_actor_record(actor, idx=idx, model_label=model_label, status="parse_failure", raw_text=raw_text))
continue
if not required and not parsed and not _empty_array_is_verified_clean(raw_text):
# Anti-refusal coverage contract: an empty array counts as a real
# "no findings" verdict only with the explicit NO_FINDINGS sentinel
# (or a bare `[]`-only response). A `[]` buried in refusal prose
# ("I cannot review this diff... []") must not enter the quorum as
# a clean PASS.
records.append(_actor_record(actor, idx=idx, model_label=model_label, status="parse_failure", raw_text=raw_text))
continue
actor_findings: List[Dict[str, Any]] = []
covered_items: set[str] = set()
for entry in parsed:

92
ouroboros/utils.py
View file

@ -69,6 +69,27 @@ def emit_log_event(
def sha256_text(s: str) -> str:
return hashlib.sha256(s.encode("utf-8")).hexdigest()
def jsonl_generation_signature(path: pathlib.Path) -> dict:
"""Identity signature of a JSONL log generation: first-line hash + size.
SSOT shared by the chat-log consolidation writer (consolidator) and the
memory reader so a rotation/rewrite is detected identically on both sides.
"""
path = pathlib.Path(path)
if not path.exists():
return {}
try:
stat = path.stat()
with path.open("r", encoding="utf-8") as handle:
first = next((line.strip() for line in handle if line.strip()), "")
return {
"first_line_sha256": hashlib.sha256(first.encode("utf-8", errors="replace")).hexdigest(),
"size": int(stat.st_size),
}
except OSError:
return {}
def read_text(path: pathlib.Path) -> str:
return path.read_text(encoding="utf-8")
@ -80,12 +101,12 @@ def write_text(path: pathlib.Path, content: str) -> None:
def atomic_write_json(
path: pathlib.Path,
payload: Dict[str, Any],
payload: Any,
*,
trailing_newline: bool = False,
fsync: bool = False,
) -> None:
"""Atomically persist a JSON object using a unique sibling temp file."""
"""Atomically persist a JSON value (object or list) via a sibling temp file."""
path = pathlib.Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
tmp_name = (
@ -169,6 +190,56 @@ def read_json_dict(path: pathlib.Path) -> Optional[Dict[str, Any]]:
return data if isinstance(data, dict) else None
def update_json_locked(
path: pathlib.Path,
mutator: Any,
*,
timeout_sec: float = 4.0,
stale_sec: float = 90.0,
) -> Dict[str, Any]:
"""Locked read-modify-write of a durable JSON dict file.
Acquires a sidecar ``<file>.lock``, re-reads the CURRENT on-disk JSON
inside the lock (so the mutator always sees the latest state, closing the
lost-update window of unlocked load→merge→write sequences), applies
``mutator(current) -> dict | None`` (``None`` aborts: the file is left
unchanged and the pre-mutation snapshot is returned), atomically writes
the result, and releases the lock.
Raises ``TimeoutError`` on lock timeout — proceeding unlocked would
silently reintroduce the exact lost-update class this helper removes.
"""
from ouroboros.platform_layer import (
acquire_exclusive_file_lock,
release_exclusive_file_lock,
)
path = pathlib.Path(path)
lock_path = path.with_name(path.name + ".lock")
lock_fd = acquire_exclusive_file_lock(
lock_path, timeout_sec=timeout_sec, stale_sec=stale_sec
)
if lock_fd is None:
raise TimeoutError(
f"update_json_locked: could not acquire {lock_path} within {timeout_sec}s"
)
try:
current = read_json_dict(path) or {}
updated = mutator(current)
if updated is None:
return current
atomic_write_json(path, updated)
return updated
finally:
release_exclusive_file_lock(lock_path, lock_fd)
def jsonl_append_lock_path(path: pathlib.Path) -> pathlib.Path:
"""Sidecar lock path shared by ``append_jsonl`` writers and log rotation."""
path_hash = hashlib.sha256(str(path.resolve()).encode("utf-8")).hexdigest()[:12]
return path.parent / f".append_jsonl_{path_hash}.lock"
def append_jsonl(path: pathlib.Path, obj: Dict[str, Any]) -> bool:
"""Append a JSON object as a line to a JSONL file (concurrent-safe).
@ -190,8 +261,7 @@ def append_jsonl(path: pathlib.Path, obj: Dict[str, Any]) -> bool:
write_retries = 3
retry_sleep_base_sec = 0.01
path_hash = hashlib.sha256(str(path.resolve()).encode("utf-8")).hexdigest()[:12]
lock_path = path.parent / f".append_jsonl_{path_hash}.lock"
lock_path = jsonl_append_lock_path(path)
lock_fd = None
lock_acquired = False
_written = False
@ -807,15 +877,11 @@ async def collect_evolution_metrics(repo_dir: str, data_dir: str | None = None)
if cache_path and new_points:
try:
cache_path.parent.mkdir(parents=True, exist_ok=True)
cache_path.write_text(
json.dumps({
"schema": 1,
"points": points_by_tag,
"updated_at": utc_now_iso(),
}, ensure_ascii=False, indent=2, sort_keys=True),
encoding="utf-8",
)
atomic_write_json(cache_path, {
"schema": 1,
"points": points_by_tag,
"updated_at": utc_now_iso(),
})
except OSError:
log.warning("Failed to write evolution metrics cache: %s", cache_path, exc_info=True)

20
ouroboros/workspace_executor.py
View file

@ -737,6 +737,26 @@ def start_service(
record.local_proc = proc
record.backend_pid = str(proc.pid)
record.backend_log_path = str(log_path)
# Write-through into the custody ledger: workspace records keep their
# own validation semantics, but the generation reaper now sees these
# too (previously swept only on clean shutdown).
try:
from ouroboros.process_custody import record_process
record_process(
pathlib.Path(getattr(ctx, "drive_root")),
pid=proc.pid,
cmd=record.cmd,
purpose=f"workspace_service:{name}",
scope="task",
owner_task_id=record.task_id,
)
except Exception:
import logging as _logging
_logging.getLogger(__name__).debug(
"workspace service custody record failed", exc_info=True
)
else:
if executor.network == "none":
_assert_docker_network_none(executor.container_name)

2
pyproject.toml
View file

@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "ouroboros"
version = "6.25.0rc3"
version = "6.26.0rc1"
description = "Self-creating AI agent with constitution, background consciousness, and persistent identity"
readme = "README.md"
license = {text = "MIT"}

1
scripts/build_repo_bundle.py
View file

@ -10,7 +10,6 @@ import os
import pathlib
import re
import subprocess
import sys
from urllib.parse import urlsplit, urlunsplit

22
scripts/download_python_standalone.sh
View file

@ -1,5 +1,5 @@
#!/bin/bash
set -e
set -euo pipefail
# Downloads python-build-standalone for macOS (arm64 + x86_64) and Linux (x86_64)
# Run from repo root: bash scripts/download_python_standalone.sh
@ -14,8 +14,10 @@ ARCH=$(uname -m)
if [ "$OS" = "Darwin" ]; then
if [ "$ARCH" = "arm64" ]; then
PLATFORM="aarch64-apple-darwin"
SHA256="e6634b06afa2ae79e664cf34174dad5d31b30117f90a01d533de3f4e9db6974e"
elif [ "$ARCH" = "x86_64" ]; then
PLATFORM="x86_64-apple-darwin"
SHA256="8d08ff2d9bb20566223f20307c8d9c31bdf020b1f1a237c2aa6efd1de651bf7b"
else
echo "Unsupported macOS architecture: $ARCH"
exit 1
@ -23,8 +25,10 @@ if [ "$OS" = "Darwin" ]; then
elif [ "$OS" = "Linux" ]; then
if [ "$ARCH" = "x86_64" ]; then
PLATFORM="x86_64-unknown-linux-gnu"
SHA256="d71df61d1cdb59af4443912da8eeca744a52e782ff5deefa966ead893235a39e"
elif [ "$ARCH" = "aarch64" ]; then
PLATFORM="aarch64-unknown-linux-gnu"
SHA256="f2916a20f3de5500df5129e37dd0a213281dea29e226ec1ffe7fecdb10955533"
else
echo "Unsupported Linux architecture: $ARCH"
exit 1
@ -43,7 +47,21 @@ echo "URL: ${URL}"
rm -rf "$DEST" _python_tmp
mkdir -p _python_tmp
curl -L --progress-bar "$URL" | tar xz -C _python_tmp
# Pinned SHA256 (from the release's SHA256SUMS): a swapped/truncated archive
# fails here instead of becoming the packaged runtime. Update the pins when
# bumping RELEASE/PY_VERSION.
curl -L --fail --progress-bar "$URL" -o _python_tmp/"$FILENAME"
if command -v shasum >/dev/null 2>&1; then
ACTUAL="$(shasum -a 256 "_python_tmp/${FILENAME}" | awk '{print $1}')"
else
ACTUAL="$(sha256sum "_python_tmp/${FILENAME}" | awk '{print $1}')"
fi
if [ "$ACTUAL" != "$SHA256" ]; then
echo "SHA256 mismatch for ${FILENAME}: expected ${SHA256}, got ${ACTUAL} — refusing to install."
exit 1
fi
tar xz -C _python_tmp -f _python_tmp/"$FILENAME"
rm -f _python_tmp/"$FILENAME"
# Archive extracts to python/ — rename to python-standalone/
mv _python_tmp/python "$DEST"

121
server.py
View file

@ -3,6 +3,7 @@
import asyncio
import json
import logging
import socket
import os
import pathlib
@ -100,18 +101,15 @@ def _restart_current_process(host: str, port: int) -> None:
_restart_current_process_impl(host, port, repo_dir=REPO_DIR, log=log)
from ouroboros.config import (
SETTINGS_DEFAULTS as _SETTINGS_DEFAULTS,
load_settings, save_settings, apply_settings_to_env as _apply_settings_to_env,
)
from ouroboros.server_runtime import (
apply_runtime_provider_defaults,
has_local_routing,
has_startup_ready_provider,
needs_local_model_autostart,
setup_remote_if_configured,
ws_heartbeat_loop,
)
from ouroboros.onboarding_wizard import build_onboarding_html
_supervisor_ready = threading.Event()
_supervisor_error: Optional[str] = None
@ -141,7 +139,7 @@ def _describe_bg_consciousness_state(requested_enabled: bool) -> dict:
elif requested_enabled and running:
status = "running"
detail = (
f"Background consciousness is idle between wakeups."
"Background consciousness is idle between wakeups."
+ (f" Next wakeup in {next_wakeup_sec}s." if next_wakeup_sec > 0 else "")
)
elif requested_enabled:
@ -212,7 +210,6 @@ def _process_bridge_updates(bridge, offset: int, ctx: Any) -> int:
st = ctx.load_state()
owner_id = st.get("owner_id")
owner_chat_id = st.get("owner_chat_id")
lowered = text.strip().lower()
is_slash_command = lowered.startswith("/")
is_external_transport = source != "web"
@ -220,10 +217,7 @@ def _process_bridge_updates(bridge, offset: int, ctx: Any) -> int:
# Global owner = primary chat for outbound notices (web on desktop, the
# first transport on headless Colab). Bound once, on the first message.
if owner_id is None and external_identity_present:
st["owner_id"] = user_id
st["owner_chat_id"] = chat_id
owner_id = user_id
owner_chat_id = chat_id
from supervisor.message_bus import log_chat
@ -255,8 +249,15 @@ def _process_bridge_updates(bridge, offset: int, ctx: Any) -> int:
"transport": transport,
"chat_id": chat_id,
})
st["last_owner_message_at"] = now_iso
ctx.save_state(st)
# Atomic owner-binding + activity stamp: the old load→(log/broadcast)→save
# span could overwrite concurrent budget/state writers with stale data.
def _stamp_owner_activity(live: dict) -> None:
if live.get("owner_id") is None and external_identity_present:
live["owner_id"] = user_id
live["owner_chat_id"] = chat_id
live["last_owner_message_at"] = now_iso
ctx.update_state(_stamp_owner_activity)
if not text and not image_base64:
continue
@ -272,10 +273,13 @@ def _process_bridge_updates(bridge, offset: int, ctx: Any) -> int:
owner_ext_id = st.get("owner_external_id")
owner_ext_chat_id = st.get("owner_external_chat_id")
if owner_ext_id is None:
st["owner_external_id"] = user_id
st["owner_external_chat_id"] = chat_id
st["owner_external_bound_at"] = now_iso
ctx.save_state(st)
def _bind_external_owner(live: dict) -> None:
if live.get("owner_external_id") is None:
live["owner_external_id"] = user_id
live["owner_external_chat_id"] = chat_id
live["owner_external_bound_at"] = now_iso
ctx.update_state(_bind_external_owner)
ctx.send_with_budget(chat_id, "✅ Owner chat registered. Send the command again to execute it.")
continue
try:
@ -504,6 +508,23 @@ def _run_supervisor(settings: dict) -> None:
_apply_settings_to_env(settings)
# Supervisor revival (e.g. settings POST after a loop death) must not leak
# the previous generation: the old BackgroundConsciousness daemon thread
# would keep burning budget unreachable by /bg stop, and the cached direct
# chat agent stays bound to the OLD event queue (messages to a dead queue).
if _consciousness is not None:
try:
_consciousness.stop()
except Exception:
log.debug("Failed to stop previous consciousness instance", exc_info=True)
_consciousness = None
try:
from supervisor import workers as _workers_mod
_workers_mod._chat_agent = None
except Exception:
log.debug("Failed to reset cached chat agent", exc_info=True)
try:
from supervisor.message_bus import init as bus_init
from supervisor.message_bus import LocalChatBridge
@ -521,7 +542,7 @@ def _run_supervisor(settings: dict) -> None:
chat_bridge=bridge,
)
from supervisor.state import init as state_init, init_state, load_state, save_state
from supervisor.state import init as state_init, init_state, load_state, save_state, update_state
from supervisor.state import append_jsonl, update_budget_from_usage, rotate_chat_log_if_needed
state_init(DATA_DIR, float(settings.get("TOTAL_BUDGET", 10.0)))
init_state()
@ -587,6 +608,14 @@ def _run_supervisor(settings: dict) -> None:
})
except Exception:
log.debug("Headless task drive prune failed", exc_info=True)
try:
from ouroboros.process_custody import reap_orphaned_processes
reaped = reap_orphaned_processes(DATA_DIR)
if reaped:
log.info("Process custody reaper killed %d orphaned process(es): %s", len(reaped), reaped)
except Exception:
log.debug("Process custody startup reap failed", exc_info=True)
try:
from ouroboros import subagent_worktrees
@ -660,6 +689,7 @@ def _run_supervisor(settings: dict) -> None:
bridge=bridge, WORKERS=WORKERS, PENDING=PENDING, RUNNING=RUNNING,
MAX_WORKERS=max_workers,
send_with_budget=send_with_budget, load_state=load_state, save_state=save_state,
update_state=update_state,
update_budget_from_usage=update_budget_from_usage, append_jsonl=append_jsonl,
enqueue_task=enqueue_task, cancel_task_by_id=cancel_task_by_id,
queue_deep_self_review_task=queue_deep_self_review_task, persist_queue_snapshot=persist_queue_snapshot,
@ -682,6 +712,7 @@ def _run_supervisor(settings: dict) -> None:
offset = 0
crash_count = 0
_last_custody_reap = [time.time()]
while not _restart_requested.is_set():
try:
rotate_chat_log_if_needed(DATA_DIR)
@ -707,6 +738,19 @@ def _run_supervisor(settings: dict) -> None:
check_scheduled_tasks()
except Exception:
log.warning("Scheduled task check failed", exc_info=True)
# Periodic custody reap: catches task-scoped processes whose owning
# task finished (or was SIGKILLed) within this server generation.
if time.time() - _last_custody_reap[0] > 600:
_last_custody_reap[0] = time.time()
try:
from ouroboros.process_custody import reap_orphaned_processes
from supervisor.queue import RUNNING as _running_tasks
reap_orphaned_processes(
DATA_DIR, running_task_ids=set(_running_tasks.keys()),
)
except Exception:
log.debug("Periodic custody reap failed", exc_info=True)
try:
from ouroboros.post_task_evolution import apply_pending_request
from supervisor import state as _pte_state
@ -727,7 +771,24 @@ def _run_supervisor(settings: dict) -> None:
crash_count += 1
log.error("Supervisor loop crash #%d: %s", crash_count, exc, exc_info=True)
if crash_count >= 3:
log.critical("Supervisor exceeded max retries.")
# Visible death: previously the loop returned with
# _supervisor_ready still set and no _supervisor_error, so
# tasks silently stopped being assigned with a healthy-looking
# /api/state. Record the failure and tell the owner.
_supervisor_error = f"Supervisor loop died after 3 consecutive crashes: {exc}"
_supervisor_ready.clear()
log.critical("Supervisor exceeded max retries: %s", _supervisor_error)
try:
st = load_state()
if st.get("owner_chat_id"):
send_with_budget(
int(st["owner_chat_id"]),
"🛑 Supervisor loop died after repeated crashes; tasks are no "
"longer being assigned. Saving settings or restarting the app "
f"will revive it. Last error: {exc}",
)
except Exception:
log.debug("Failed to notify owner about supervisor death", exc_info=True)
return
time.sleep(min(30, 2 ** crash_count))
_supervisor_thread = None
@ -913,6 +974,18 @@ async def lifespan(app):
init_global_supervisor(lifespan_drive_root)
host_service_app = create_host_service_app(lifespan_drive_root)
host_port = host_service_port()
# Probe the port first: uvicorn's Server.startup() calls sys.exit(1) on a
# bind error, and SystemExit raised inside an asyncio task escapes
# run_forever and takes down the WHOLE main server (a stale prior
# instance still holding the port is exactly the realistic trigger).
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as _probe:
_probe.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
_probe.bind((DEFAULT_HOST_SERVICE_HOST, host_port))
except OSError as bind_exc:
raise RuntimeError(
f"Host Service port {host_port} is busy: {bind_exc}"
) from bind_exc
host_service_config = uvicorn.Config(
host_service_app,
host=DEFAULT_HOST_SERVICE_HOST,
@ -1080,6 +1153,14 @@ app.app.state.default_port = DEFAULT_PORT # type: ignore[attr-defined]
app.app.state.start_supervisor_if_needed = _start_supervisor_if_needed # type: ignore[attr-defined]
_ACTUAL_BOUND_PORT: Optional[int] = None
def _actual_bound_port() -> int:
"""Port the server actually bound (set in main(); DEFAULT_PORT before that)."""
return _ACTUAL_BOUND_PORT if _ACTUAL_BOUND_PORT else DEFAULT_PORT
def _emergency_process_cleanup(*, port_sweep: bool = True) -> None:
"""Kill child processes, workers, companions, and runtime port holders."""
try:
@ -1128,8 +1209,10 @@ def _emergency_process_cleanup(*, port_sweep: bool = True) -> None:
except Exception:
pass
if port_sweep:
kill_process_on_port(DEFAULT_PORT)
kill_process_on_port(8766)
# Sweep the ACTUALLY bound port (find_free_port may have moved off
# DEFAULT_PORT); the old hardcoded 8765/8766 pair could kill an
# unrelated process on a custom-port install.
kill_process_on_port(_actual_bound_port())
try:
from ouroboros.extension_companion import panic_kill_all
from ouroboros.gateway.host_service import host_service_port
@ -1159,6 +1242,8 @@ def main() -> int:
actual_port = find_free_port(args.host, args.port)
if actual_port != args.port:
log.info("Port %d busy on %s, using %d instead", args.port, args.host, actual_port)
global _ACTUAL_BOUND_PORT
_ACTUAL_BOUND_PORT = actual_port
write_port_file(PORT_FILE, actual_port)
log.info("Starting Ouroboros server on %s:%d", args.host, actual_port)
config = uvicorn.Config(

108
supervisor/events.py
View file

@ -2,16 +2,14 @@
from __future__ import annotations
import json
import logging
import pathlib
import subprocess
import sys
import time
import uuid
from typing import Any, Dict, Optional
from ouroboros.utils import truncate_for_log, utc_now_iso
from ouroboros.utils import append_jsonl, truncate_for_log, utc_now_iso
from ouroboros.config import get_max_active_subagents_per_root, get_max_subagent_depth
from ouroboros.tool_capabilities import ACTING_SUBAGENT_MODE, LOCAL_READONLY_SUBAGENT_MODE
from ouroboros.contracts.task_constraint import VALID_WRITE_SURFACES
@ -348,27 +346,28 @@ def _handle_llm_usage(evt: Dict[str, Any], ctx: Any) -> None:
usage: Dict[str, Any] = usage_raw if isinstance(usage_raw, dict) else {}
# Normalize usage across loop.py, web_search, and claude_code_edit producers.
prompt_tokens = int(
usage.get("prompt_tokens")
or usage.get("input_tokens")
or evt.get("prompt_tokens")
or 0
# Tolerant coercion: one malformed token field must not raise and drop the
# whole round from the budget ledger and events.jsonl (the exception would
# be swallowed by dispatch_event and the cost silently lost).
def _tolerant_int(*candidates: Any) -> int:
for value in candidates:
if value in (None, ""):
continue
try:
return int(float(value))
except (TypeError, ValueError):
log.warning("llm_usage: non-numeric token field %r ignored", value)
return 0
prompt_tokens = _tolerant_int(
usage.get("prompt_tokens"), usage.get("input_tokens"), evt.get("prompt_tokens")
)
completion_tokens = int(
usage.get("completion_tokens")
or usage.get("output_tokens")
or evt.get("completion_tokens")
or 0
completion_tokens = _tolerant_int(
usage.get("completion_tokens"), usage.get("output_tokens"), evt.get("completion_tokens")
)
cached_tokens = int(
usage.get("cached_tokens")
or evt.get("cached_tokens")
or 0
)
cache_write_tokens = int(
usage.get("cache_write_tokens")
or evt.get("cache_write_tokens")
or 0
cached_tokens = _tolerant_int(usage.get("cached_tokens"), evt.get("cached_tokens"))
cache_write_tokens = _tolerant_int(
usage.get("cache_write_tokens"), evt.get("cache_write_tokens")
)
prompt_cache_ttl = str(
usage.get("prompt_cache_ttl")
@ -395,7 +394,6 @@ def _handle_llm_usage(evt: Dict[str, Any], ctx: Any) -> None:
}
ctx.update_budget_from_usage(usage_for_budget)
from ouroboros.utils import utc_now_iso, append_jsonl
try:
append_jsonl(ctx.DRIVE_ROOT / "logs" / "events.jsonl", {
"ts": evt.get("ts", utc_now_iso()),
@ -545,7 +543,6 @@ def _handle_task_done(evt: Dict[str, Any], ctx: Any) -> None:
final_task_result = {}
# Persist here so send_message reaches the UI before task_done collapses the card.
from ouroboros.utils import utc_now_iso, append_jsonl
outcome_axes = normalize_outcome_axes({**evt, **(final_task_result if isinstance(final_task_result, dict) else {})})
reason_code = final_task_result.get("reason_code") or evt.get("reason_code")
artifact_status = final_task_result.get("artifact_status") or evt.get("artifact_status")
@ -588,7 +585,6 @@ def _handle_task_done(evt: Dict[str, Any], ctx: Any) -> None:
log.warning("Failed to log task_done to events.jsonl", exc_info=True)
if task_type == "evolution":
st = ctx.load_state()
# Meaningful evolution work has non-trivial cost plus at least one round.
# eff_* falls back to the persisted (reconstructed) result on abnormal
# termination so a zeroed terminal event cannot understate the tally or
@ -644,29 +640,39 @@ def _handle_task_done(evt: Dict[str, Any], ctx: Any) -> None:
if replayed_evolution_terminal:
pass
elif not failed_by_axes and rounds >= 1:
st["evolution_consecutive_failures"] = 0
ctx.save_state(st)
from supervisor.state import update_state
update_state(lambda live: live.update(evolution_consecutive_failures=0))
else:
failures = int(st.get("evolution_consecutive_failures") or 0) + 1
st["evolution_consecutive_failures"] = failures
ctx.save_state(st)
from supervisor.state import update_state
failures_box: Dict[str, int] = {}
def _bump_failures(live: Dict[str, Any]) -> None:
failures_box["n"] = int(live.get("evolution_consecutive_failures") or 0) + 1
live["evolution_consecutive_failures"] = failures_box["n"]
update_state(_bump_failures)
ctx.append_jsonl(
ctx.DRIVE_ROOT / "logs" / "supervisor.jsonl",
{
"ts": utc_now_iso(),
"type": "evolution_task_failure_tracked",
"task_id": task_id,
"consecutive_failures": failures,
"consecutive_failures": failures_box.get("n", 0),
"cost_usd": cost,
"rounds": rounds,
},
)
try:
cur = ctx.load_state()
if cur.get("post_task_autostop"):
cur["evolution_mode_enabled"] = False
cur["post_task_autostop"] = False
ctx.save_state(cur)
from supervisor.state import update_state
def _consume_autostop(live: Dict[str, Any]) -> None:
if live.get("post_task_autostop"):
live["evolution_mode_enabled"] = False
live["post_task_autostop"] = False
update_state(_consume_autostop)
except Exception:
log.debug("Post-task evolution autostop failed", exc_info=True)
@ -710,9 +716,13 @@ def _handle_task_done(evt: Dict[str, Any], ctx: Any) -> None:
"artifact_status": str(effective_result.get("artifact_status") or ""),
},
)
ctx.RUNNING.pop(str(task_id), None)
if wid in ctx.WORKERS and ctx.WORKERS[wid].busy_task_id == task_id:
ctx.WORKERS[wid].busy_task_id = None
from supervisor.queue import _queue_lock
with _queue_lock:
if task_id:
ctx.RUNNING.pop(str(task_id), None)
if wid in ctx.WORKERS and ctx.WORKERS[wid].busy_task_id == task_id:
ctx.WORKERS[wid].busy_task_id = None
ctx.persist_queue_snapshot(reason="task_done")
try:
ctx.bridge.push_log(task_done_event)
@ -1532,14 +1542,17 @@ def _handle_toggle_evolution(evt: Dict[str, Any], ctx: Any) -> None:
if st.get("owner_chat_id"):
ctx.send_with_budget(int(st["owner_chat_id"]), block)
return
st = ctx.load_state()
st["evolution_mode_enabled"] = enabled
if enabled:
st["evolution_consecutive_failures"] = 0
# Symmetry with the owner /evolve path: an explicit toggle must not inherit a
# stale post-task one-shot autostop that would disable the campaign after one cycle.
st["post_task_autostop"] = False
ctx.save_state(st)
from supervisor.state import update_state
def _toggle_evolution(live: Dict[str, Any]) -> None:
live["evolution_mode_enabled"] = enabled
if enabled:
live["evolution_consecutive_failures"] = 0
# Symmetry with the owner /evolve path: an explicit toggle must not inherit a
# stale post-task one-shot autostop that would disable the campaign after one cycle.
live["post_task_autostop"] = False
st = update_state(_toggle_evolution)
try:
from supervisor.queue import pause_evolution_campaign, start_evolution_campaign
@ -1648,7 +1661,6 @@ def _handle_send_video(evt: Dict[str, Any], ctx: Any) -> None:
def _handle_owner_message_injected(evt: Dict[str, Any], ctx: Any) -> None:
"""Log owner injections so health checks can detect duplicate processing."""
from ouroboros.utils import utc_now_iso
try:
ctx.append_jsonl(ctx.DRIVE_ROOT / "logs" / "events.jsonl", {
"ts": evt.get("ts", utc_now_iso()),

9
supervisor/git_ops.py
View file

@ -177,9 +177,12 @@ def _read_update_intent() -> Dict[str, Any]:
def _write_update_intent(payload: Dict[str, Any]) -> None:
# Atomic: a torn marker would make the next restart silently skip the
# prepared managed update (reader fails closed on parse errors).
from ouroboros.utils import atomic_write_json
path = _update_intent_marker_path()
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(payload, ensure_ascii=True, indent=2) + "\n", encoding="utf-8")
atomic_write_json(path, payload, trailing_newline=True)
def _clear_update_intent() -> None:
@ -1030,7 +1033,7 @@ def safe_restart(
if t2["ok"]:
return True, f"OK: fell back to {BRANCH_STABLE}"
return False, f"Both branches failed import (dev and stable)"
return False, "Both branches failed import (dev and stable)"
def list_versions(max_count: int = 50) -> List[Dict[str, Any]]:

Some files were not shown because too many files have changed in this diff Show more