diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2fa43d09e..b6e56eb80 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,12 +1,12 @@ # Ouroboros CI — Five-tier cross-platform testing and release pipeline # # Tier 1 (Quick): Push to ouroboros or PR targeting ouroboros → Ubuntu-only tests -# Tier 2 (Full): Every PR → Windows/macOS; stable / manual / tag → full 3-OS matrix -# Tier 2.5 (Integration): Push to main / ouroboros / ouroboros-stable / manual / tag → Real-provider tests (~2 min) +# Tier 2 (Full): Every PR and every ouroboros push → Windows/macOS; stable / manual / tag → full 3-OS matrix +# Tier 2.5 (Integration): Manual / tag (branch pushes: provider-canary-push.yml) → Real-provider tests, reusable provider-canary.yml (~2 min) # Tier 2.6 (Skill smoke): Push to ouroboros-stable / manual / tag → LIVE OuroborosHub official-skill install smoke (3-OS, ~5 min) + review→grant→enable-persistence flow on one cheap reviewer slot (ubuntu-only step, OPENROUTER_API_KEY, ~$1.2/run) # Tier 3 (Build+Release): Tag v* → PyInstaller + GitHub Release (~15 min) # Scheduled (System E2E): Daily cron / manual → KEYLESS tests/system_e2e/ scenario lane (Ubuntu, ≤40 min) -# Scheduled (Live E2E): Nightly cron 03:17 UTC (ouroboros tip) / dispatch input e2e_live=true → PAID devtools/e2e_live stand, $30 cap, OUROBOROS_E2E_LIVE_OPENROUTER_KEY (skipped honestly without it) +# Manual (Live E2E): Dispatch input e2e_live=true only (no schedule) → PAID devtools/e2e_live stand, $30 cap, OUROBOROS_E2E_LIVE_OPENROUTER_KEY (skipped honestly without it) # # Tier 2.5 requires OPENROUTER_API_KEY / OPENAI_API_KEY / ANTHROPIC_API_KEY. # Optional MiniMax / DeepSeek / Cloud.ru / GigaChat rows run when their diff --git a/.github/workflows/provider-canary-push.yml b/.github/workflows/provider-canary-push.yml index 3a2f94aaa..c8447fb55 100644 --- a/.github/workflows/provider-canary-push.yml +++ b/.github/workflows/provider-canary-push.yml @@ -3,7 +3,7 @@ # The canaries call third-party models, so a red run here can mean a provider # misbehaved while the repository's code is fine. They live in their own # workflow so that outcome carries its own badge and never reddens `ci.yml`, -# which reports the code. The run frequency per push is unchanged: this trigger +# which reports the code. Each matching push starts one paid run: this trigger # carries the same branches and the same path filter as the `ci.yml` push # trigger, and `ci.yml` calls the same reusable job only for manual runs and # `v*` tags, where the release chain requires it. diff --git a/docs/architecture/08-git-branching-ci-and-build.md b/docs/architecture/08-git-branching-ci-and-build.md index cce71afb3..866959109 100644 --- a/docs/architecture/08-git-branching-ci-and-build.md +++ b/docs/architecture/08-git-branching-ci-and-build.md @@ -16,10 +16,10 @@ The local `ouroboros-stable` ref is also a recovery fallback maintained by expli | Android source/build proof | `android-test` | Every non-scheduled run; `android/tests`, release tests and disposable-key APK compilation; no publisher secrets; no physical proof. | | Android emulator proof | `android-emulator-smoke` | Tags or verified `android/` changes from fork-safe `android-test` base/head comparison; API 26/29/30/33/36 x86_64 install/start and same-UID PackageInstaller readback; no publisher secrets; no physical proof. | | Browser consumer proof | `ui-browser.yml`, called by `ci.yml` and `ui-browser-push.yml` | Reusable lane: PRs, full manual runs and `v*` tags via `ci.yml`; every `ouroboros` push and UI-only manual selection via `ui-browser-push.yml`. No path filter: Makefile-, spec- and requirements-only merges run too. Full `ui_browser` lane with engine-availability, collection and executed-node guards; full manual runs and tags also keep browser-tool coverage. `scripts/browser_ci_scope.py` skips only a complete-documentation diff; an unreadable base runs the lane. No schedule, no new cron, no provider secrets, no added default local pytest lane. | -| Scheduled lanes | `system-e2e-mock` (keyless; daily 04:37 UTC, manual runs, `v*` tags — on the release bar), `e2e-live` (`OUROBOROS_E2E_LIVE_OPENROUTER_KEY`, `$30` cap) | `e2e-live` fires only on its own nightly cron (03:17 UTC, seeding the `ouroboros` branch tip rather than the default branch the schedule fires on) or a dispatch that opts in through the `e2e_live` input; without the secret it is one summary line and green, never a pretend run | -| Ordinary desktop tests | `full-test` (no secrets) | Every PR into `ouroboros` runs Windows/macOS alongside Ubuntu quick-test; stable pushes, manual runs and `v*` tags retain the full three-OS matrix | +| Scheduled and opt-in lanes | `system-e2e-mock` (keyless; daily 04:37 UTC, manual runs, `v*` tags — on the release bar), `e2e-live` (`OUROBOROS_E2E_LIVE_OPENROUTER_KEY`, `$30` cap) | `e2e-live` runs on no schedule: only a dispatch that opts in through the `e2e_live` input starts it, seeding the dispatched sha; without the secret it is one summary line and green, never a pretend run | +| Ordinary desktop tests | `full-test` (no secrets) | Every PR into `ouroboros` and every `ouroboros` push that passes the path filter runs Windows/macOS alongside Ubuntu quick-test, so the landed commit shows which merge broke a desktop system; stable pushes, manual runs and `v*` tags retain the full three-OS matrix | | Trusted skill matrix | `skill-smoke` (`OPENROUTER_API_KEY`) | `ouroboros-stable` pushes, manual runs, `v*` tags — never pull requests | -| Trusted provider run | `integration-test` | provider secrets; `main`/`ouroboros`/`ouroboros-stable` pushes, manual runs and `v*` tags (the release chain needs it) — never pull requests | +| Trusted provider run | `integration-test`, body in `provider-canary.yml` | provider secrets; manual runs and `v*` tags via `ci.yml` (the release chain needs it), `main`/`ouroboros`/`ouroboros-stable` pushes via `provider-canary-push.yml` (same path filter, its own "Provider canaries" result, so a provider outage does not redden `ci.yml`) — never pull requests | | Tag-only gates and release chain | `marker-guards`, `docker-ui-smoke`, `docker-portable-test` (manual runs or tags, no secrets); `release-preflight` always waits for `full-test` + `integration-test` + `system-e2e-mock`, preserves tag/VERSION metadata, then fails closed when any required result is not `success`; a valid tag may run the desktop `build` diagnostically so later artifact failures remain visible, while `release` still requires a successful preflight. `android-build` retains its needs on `android-test` + `android-emulator-smoke` + preflight, so failed Android proofs exclude both optional Android assets; `release` waits for both builds but requires success only from the desktop build, `release-preflight`, `skill-smoke`, `ui-smoke` and these three smoke gates; workflow cancellation is respected. `vendor-package-smoke` needs `build` and stays informational | tag-triggered; a reproducible provider-contract failure blocks publication but does not hide desktop build diagnostics, and a typed inconclusive provider outage does not block | Quick and full jobs run blocking `size_ratchet` pytest steps — the only enforcement of repository size gates (local runs exclude the marker and warn): manifest exactness on the tip plus the pairwise shrink-only transition against the event base in `OURO_SIZE_RATCHET_BASE_REF`. An unresolvable base degrades to the tip's parent manifest verified against the parent's own tree, never a skip (an unreadable parent fails closed), while a resolvable base without a manifest fails closed, because accepting a copied manifest would allow debt laundering. Both jobs also run the browser-module suite (`cd web && node --test tests/*.test.js`), the node lane the hermetic commit gate executes through `ouroboros/preflight_node.py`. Secret-bearing skill review runs before any step that imports downloaded plugin code — untrusted payload code must never share a process with provider credentials — and a missing required key is red rather than skipped. diff --git a/docs/development/06-rules-by-change-class.md b/docs/development/06-rules-by-change-class.md index 74169f37f..acc83093e 100644 --- a/docs/development/06-rules-by-change-class.md +++ b/docs/development/06-rules-by-change-class.md @@ -83,7 +83,7 @@ Admit an attempt only while settled spend plus in-flight reservations plus its o #### Reports, focused verification and CI -Run roots are append-only outside `repo/` and live `data/`; the focused contracts are the `tests/test_e2e_live_*` modules plus `tests/test_server_runner_absorb_wait.py`, and the `e2e-live` CI job runs only on its nightly cron or an explicit `e2e_live=true` dispatch — see devtools/e2e_live/README.md#reports-focused-verification-and-ci. +Run roots are append-only outside `repo/` and live `data/`; the focused contracts are the `tests/test_e2e_live_*` modules plus `tests/test_server_runner_absorb_wait.py`, and the `e2e-live` CI job runs only on an explicit `e2e_live=true` dispatch — see devtools/e2e_live/README.md#reports-focused-verification-and-ci. ### Light mode and external deliverables diff --git a/ouroboros/tools/git.py b/ouroboros/tools/git.py index 0c68c21c3..908adaa1e 100644 --- a/ouroboros/tools/git.py +++ b/ouroboros/tools/git.py @@ -935,7 +935,8 @@ def _check_ci_status_after_push(repo_dir: pathlib.Path) -> str: return "\n\n⏳ CI: Run not yet registered — check GitHub Actions in ~30s." # One push starts several workflows (code, browser lane, provider # canaries), so each is reported under its own name and none stands for - # the others. The API lists newest first: a re-run supersedes its original. + # the others. The API lists newest first: the first row per workflow is + # the one for the latest push of this commit. latest: Dict[str, dict] = {} for run in runs: latest.setdefault(str(run.get("name") or run.get("path") or "workflow"), run) @@ -945,10 +946,12 @@ def _check_ci_status_after_push(repo_dir: pathlib.Path) -> str: for name, run in latest.items() } summary = "; ".join(f"{name}: {state}" for name, state in states.items()) - failed = next((name for name, run in latest.items() - if run.get("status") == "completed" and states[name] != "success"), None) + settled = ("success", "skipped", "neutral") # A workflow whose jobs all skip is not a failure. + red = [name for name, run in latest.items() + if run.get("status") == "completed" and states[name] not in settled] + failed = red[0] if red else None if failed is None: - if all(state == "success" for state in states.values()): + if all(state in settled for state in states.values()): return f"\n\n✅ CI: registered push runs passed for this commit — {summary}." return f"\n\n⏳ CI: push runs in progress — {summary}. Check GitHub Actions for results." completed, conclusion = latest[failed], states[failed] @@ -972,7 +975,7 @@ def _check_ci_status_after_push(repo_dir: pathlib.Path) -> str: pass # Fall back to generic summary — run_number/html_url still surfaced below if conclusion == "failure": return ( - f"\n\n⚠️ CI STATUS: {failed} FAILED for this commit (run #{run_number})\n" + f"\n\n⚠️ CI STATUS: {', '.join(red)} FAILED for this commit (run #{run_number})\n" f" Workflows: {summary}\n" f" Failed: {failed_summary}\n" f" Fix: investigate the failed jobs; push a fix commit when this commit caused them.\n" diff --git a/scripts/run_external_review.py b/scripts/run_external_review.py index 04b9fe7cd..4e2f470e2 100755 --- a/scripts/run_external_review.py +++ b/scripts/run_external_review.py @@ -108,6 +108,7 @@ _REVIEW_SUBSTRATE_PATHS = frozenset({ }) _RELEASE_MACHINERY_PATHS = frozenset({ ".github/workflows/ci.yml", + ".github/workflows/provider-canary.yml", ".github/workflows/provider-canary-push.yml", # ci.yml's canary job "build.sh", "build_linux.sh", "build_windows.ps1", diff --git a/tests/test_ci_diagnostic_workflows.py b/tests/test_ci_diagnostic_workflows.py index ff58b1f0c..a2be4ca77 100644 --- a/tests/test_ci_diagnostic_workflows.py +++ b/tests/test_ci_diagnostic_workflows.py @@ -92,6 +92,12 @@ def _canary_wiring_faults(ci, shared, push): faults.append(f"{label} does not pass exactly the eight provider secrets by name") if caller.get("permissions") != {"contents": "read"}: faults.append(f"{label} widens the called job's permissions") + # Exact key sets: a condition or a group on any of the three jobs changes how often the paid job runs. + for label, job, keys in (("ci.yml", ci["jobs"]["integration-test"], {"if", "uses", "secrets", "permissions"}), + (CANARY_PUSH, push["jobs"]["integration-test"], {"uses", "secrets", "permissions"}), + (CANARY, shared["jobs"]["integration-test"], {"runs-on", "steps"})): + if set(job) != keys: + faults.append(f"{label} canary job carries keys other than {sorted(keys)}") declared = {name: {"required": False} for name in PROVIDER_SECRETS} if _triggers(shared) != {"workflow_call": {"secrets": declared}}: faults.append("the shared job is not call-only with eight optional secrets") @@ -136,6 +142,11 @@ CANARY_DRIFTS = { "caller leaves the shared job": lambda w: w.ci["jobs"]["integration-test"].update(uses="./other.yml"), "shared job loses its read-only default": lambda w: w.shared.pop("permissions"), "caller widens permissions": lambda w: w.push["jobs"]["integration-test"].update(permissions="write-all"), + "wrapper job condition": lambda w: w.push["jobs"]["integration-test"].update( + {"if": "github.ref == 'refs/heads/ouroboros-stable'"}), + "shared job condition": lambda w: w.shared["jobs"]["integration-test"].update( + {"if": "github.event_name == 'workflow_dispatch'"}), + "code-workflow caller concurrency": lambda w: w.ci["jobs"]["integration-test"].update(concurrency="canary"), } diff --git a/tests/test_netres_git_bounded.py b/tests/test_netres_git_bounded.py index 7768159aa..a7bf62b36 100644 --- a/tests/test_netres_git_bounded.py +++ b/tests/test_netres_git_bounded.py @@ -444,9 +444,20 @@ def test_ci_note_attributes_a_failure_to_its_workflow(monkeypatch): assert " Failed: integration-test / integration-test → Run integration tests\n" in note assert note.endswith(" URL: https://example.test/2") - # A re-run supersedes its original: the API lists the newest run first. + # The same commit pushed again: the API lists the newest run of a workflow first. rerun = _ci_note(monkeypatch, [{"name": "Provider canaries", **_DONE}, *runs]) assert rerun.startswith("\n\n✅") and "failure" not in rerun + # A finished failure is reported while another workflow still runs, and every red workflow is named. + racing = _ci_note(monkeypatch, [ + {"name": "CI", "status": "in_progress", "conclusion": None}, + {"name": "Provider canaries", "status": "completed", "conclusion": "failure"}, + {"name": "UI browser (ouroboros push)", "status": "completed", "conclusion": "failure"}, + ]) + assert "⚠️ CI STATUS: Provider canaries, UI browser (ouroboros push) FAILED" in racing and "⏳" not in racing + assert "CI: in progress" in racing + skipped = _ci_note(monkeypatch, [{"name": "CI", **_DONE}, + {"name": "Sync mirror", "status": "completed", "conclusion": "skipped"}]) + assert skipped.startswith("\n\n✅") and "Sync mirror: skipped" in skipped cancelled = _ci_note(monkeypatch, [{"name": "CI", "status": "completed", "conclusion": "cancelled"}]) assert "⚠️ CI STATUS: CI CANCELLED for this commit (run #1)" in cancelled diff --git a/tests/test_reference_book_budgets.py b/tests/test_reference_book_budgets.py index bc458ed65..d9467251c 100644 --- a/tests/test_reference_book_budgets.py +++ b/tests/test_reference_book_budgets.py @@ -277,7 +277,10 @@ CHAPTER_BYTE_BUDGETS: dict[str, int] = { # 22500 -> 22800 (measured 22774): the run-cancellation rule (which CI runs a newer pull-request # head cancels, and why re-runs keep their own group), a workflow contract the chapter had no # text for; the pointer sentence beside it was reworded in place. - "docs/architecture/08-git-branching-ci-and-build.md": 22800, + # 22800 -> 23200 (measured 23005): the CI table names where the provider canaries run for each + # event (the shared body and its push wrapper) and that landed pushes get the desktop matrix; + # the live-stand row shrank, the other two rows describe wiring the chapter did not have. + "docs/architecture/08-git-branching-ci-and-build.md": 23200, # 12405 -> 14400 (issue #1142): the ordinary-close paragraph gains the mechanism the chapter had # no text for — graceful stop signals the server PID only, the server half (stop event at the # signal, bounded uvicorn drain) is self-sufficient against an old group-SIGTERM launcher. diff --git a/tests/test_runtime_mode_core.py b/tests/test_runtime_mode_core.py index 6dded4be4..b5ce5d034 100644 --- a/tests/test_runtime_mode_core.py +++ b/tests/test_runtime_mode_core.py @@ -655,12 +655,14 @@ def test_provider_canary_workflows_are_protected_exactly_like_ci_yml(): branch-push trigger live in two workflow files beside ci.yml; an inventory naming only the parent would leave the release canary editable.""" from ouroboros.runtime_mode_policy import RELEASE_INVARIANT_PATHS + from scripts.run_external_review import _RELEASE_MACHINERY_PATHS parent = protected_path_category(".github/workflows/ci.yml") for name in ("provider-canary.yml", "provider-canary-push.yml"): path = f".github/workflows/{name}" assert (REPO / path).is_file(), path assert path in RELEASE_INVARIANT_PATHS, path + assert path in _RELEASE_MACHINERY_PATHS, path # The contributor label follows the body too. assert protected_path_category(path) == protected_path_category(f"./{path}") == parent, path # The category follows the listed files, not the directory. assert protected_path_category(".github/workflows/unlisted.yml") == ""