diff --git a/README.md b/README.md index f4d93b869..9e65d885d 100644 --- a/README.md +++ b/README.md @@ -138,10 +138,10 @@ Ouroboros has reproducible self-reported state-of-the-art results on Terminal-Be | Benchmark | Model | Ouroboros | Comparison | Status | Evidence | |-----------|-------|----------:|------------|--------|----------| -| Terminal-Bench 2.1 | Claude Opus-5 high | **86.74%** after zeroing one disclosed reward-hack trial (raw: 86.97%) | Claude Code + Fable 5: 83.8% | Self-reported, submission closed without merge (September 19, 2026) | [submission](https://github.com/harbor-framework/terminal-bench-2-1/pull/175) · [run](https://hub.harborframework.com/jobs/2b145543-edeb-4a3b-b46f-4800310f1182) | +| Terminal-Bench 2.1 | Claude Opus-5 high | **86.74%** after zeroing one disclosed reward-hack trial (raw: 86.97%) | Claude Code + Fable 5: 83.8% | Self-reported; community submissions closed | [submission](https://github.com/harbor-framework/terminal-bench-2-1/pull/175) · [run](https://hub.harborframework.com/jobs/2b145543-edeb-4a3b-b46f-4800310f1182) | | Terminal-Bench 2.1 | Claude Opus-4.8 high | **80.22%** | Claude Code: 78.9% | Self-reported, public run | [run](https://hub.harborframework.com/jobs/4b8e244f-8ab0-4d28-8218-7cf346282faa) | | Terminal-Bench 2.1 | GPT-5.5 | **84.3%** | Codex CLI: 83.1% | Self-reported, public run | [run](https://hub.harborframework.com/jobs/f02fd019-23e1-495f-af0a-ebd9a65f3079) | -| Terminal-Bench 2.1 | Grok-4.5 | **84.94%** after a reward-hack audit | Cursor CLI: 79.3% · Hermes: 77.53% | Self-reported, submission closed without merge (September 19, 2026) | [submission](https://github.com/harbor-framework/terminal-bench-2-1/pull/146) | +| Terminal-Bench 2.1 | Grok-4.5 | **84.94%** after a reward-hack audit | Cursor CLI: 79.3% · Hermes: 77.53% | Self-reported; community submissions closed | [submission](https://github.com/harbor-framework/terminal-bench-2-1/pull/146) | | OSWorld-Verified | Claude Opus-5 | **90.69%** | previous best on the public board: 90.19% | Self-reported, full traces | [full traces](https://huggingface.co/datasets/razzant/ouroboros-osworld-verified-opus5) | | OSWorld-Verified | Claude Sonnet-4.6 | **83.27%** | Pointer: 81.45% | Self-reported, full traces | [full traces](https://huggingface.co/datasets/razzant/ouroboros-osworld-verified-sonnet46) | | CL-Bench | Claude Sonnet-4.6 | **0.2301, rank 1** | previous top: 0.1960 | Self-reported, submission open | [submission](https://github.com/pgasawa/continual-learning-bench/pull/10) · [full traces](https://huggingface.co/datasets/razzant/ouroboros-clbench-traces) | diff --git a/docs/benchmarks/index.html b/docs/benchmarks/index.html index fc3a90f31..21078a6de 100644 --- a/docs/benchmarks/index.html +++ b/docs/benchmarks/index.html @@ -36,15 +36,15 @@

BENCHMARKS / PUBLIC EVIDENCE

Scores, status, and evidence.

-

These results are self-reported, and each row states what evidence is public. The GAIA row is marked while its scrubbed trace capsule is pending. Upstream submission status stays visible; the Terminal-Bench 2.1 submissions were closed without merge on September 19, 2026. The public runs and supported results remain available.

+

These results are self-reported, and each row states what evidence is public. The GAIA row is marked while its scrubbed trace capsule is pending. Publication does not wait for a leaderboard maintainer to merge submissions. Terminal-Bench 2.1 closed community submissions on September 19, 2026.

- + - + diff --git a/docs/index.html b/docs/index.html index d98b3a1ba..2fc4c94ff 100644 --- a/docs/index.html +++ b/docs/index.html @@ -96,7 +96,7 @@

02 / EVIDENCE

Measured against the strongest harnesses.

Ouroboros has reproducible self-reported state-of-the-art results on Terminal-Bench 2.1, OSWorld-Verified, and CL-Bench. In those model-matched results, it leads Codex, Claude Code, Cursor, and Hermes. The public SWE-bench Pro matched pair is a statistical tie with Codex CLI. A separate GAIA result is marked as awaiting its scrubbed trace capsule.

Publication statusSelf-reported results are published with evidence and each submission's status.Read the evidence table

-
Self-reported Terminal-Bench 2.1 results: Ouroboros 86.74% with Claude Opus-5 after a disclosed reward-hack correction, ahead of Claude Code, Codex, Cursor, and Hermes on matched models
Terminal-Bench 2.1 · 86.74% corrected · self-reportedClosed submission (not merged) · Full run
+
Self-reported Terminal-Bench 2.1 results: Ouroboros 86.74% with Claude Opus-5 after a disclosed reward-hack correction, ahead of Claude Code, Codex, Cursor, and Hermes on matched models
Terminal-Bench 2.1 · 86.74% corrected · self-reportedSubmission · Full run
Self-reported OSWorld-Verified results: Ouroboros 90.69% with Claude Opus-5 and 83.27% with Claude Sonnet-4.6
OSWorld-Verified · 90.69% · self-reportedFull traces
Self-reported CL-Bench result: Ouroboros normalized reward 0.2301 with Claude Sonnet-4.6, rank 1
CL-Bench · 0.2301 · self-reported rank #1Open submission · Full traces
diff --git a/site/benchmarks/index.html b/site/benchmarks/index.html index 5206a7a3d..837ac51dd 100644 --- a/site/benchmarks/index.html +++ b/site/benchmarks/index.html @@ -36,15 +36,15 @@

BENCHMARKS / PUBLIC EVIDENCE

Scores, status, and evidence.

-

These results are self-reported, and each row states what evidence is public. The GAIA row is marked while its scrubbed trace capsule is pending. Upstream submission status stays visible; the Terminal-Bench 2.1 submissions were closed without merge on September 19, 2026. The public runs and supported results remain available.

+

These results are self-reported, and each row states what evidence is public. The GAIA row is marked while its scrubbed trace capsule is pending. Publication does not wait for a leaderboard maintainer to merge submissions. Terminal-Bench 2.1 closed community submissions on September 19, 2026.

BenchmarkModelOuroborosComparisonStatus and evidence
Terminal-Bench 2.1Claude Opus-5 high86.74% after zeroing one disclosed reward-hack trial (raw: 86.97%)Claude Code + Fable 5: 83.8%Self-reported with a closed submission (not merged) and a public run.
Terminal-Bench 2.1Claude Opus-5 high86.74% after zeroing one disclosed reward-hack trial (raw: 86.97%)Claude Code + Fable 5: 83.8%Self-reported with a submission and a public run; community submissions closed.
Terminal-Bench 2.1Claude Opus-4.8 high80.22%Claude Code: 78.9%Self-reported with a public run.
Terminal-Bench 2.1GPT-5.584.3%Codex CLI: 83.1%Self-reported with a public run.
Terminal-Bench 2.1Grok-4.584.94%Cursor CLI: 79.3%. Hermes: 77.53%.Self-reported after a reward-hack audit, with a closed submission (not merged).
Terminal-Bench 2.1Grok-4.584.94%Cursor CLI: 79.3%. Hermes: 77.53%.Self-reported after a reward-hack audit, with a submission; community submissions closed.
OSWorld-VerifiedClaude Opus-590.69%Previous best on the public board: 90.19%Self-reported with public task evidence.
OSWorld-VerifiedClaude Sonnet-4.683.27%Pointer: 81.45%Self-reported with public task evidence.
CL-BenchClaude Sonnet-4.60.2301, rank 1Previous top: 0.1960Self-reported with an open submission and full traces.
- + - + diff --git a/site/index.html b/site/index.html index 9da8e17ff..d96dd3cb2 100644 --- a/site/index.html +++ b/site/index.html @@ -95,7 +95,7 @@

02 / EVIDENCE

Measured against the strongest harnesses.

Ouroboros has reproducible self-reported state-of-the-art results on Terminal-Bench 2.1, OSWorld-Verified, and CL-Bench. In those model-matched results, it leads Codex, Claude Code, Cursor, and Hermes. The public SWE-bench Pro matched pair is a statistical tie with Codex CLI. A separate GAIA result is marked as awaiting its scrubbed trace capsule.

Publication statusSelf-reported results are published with evidence and each submission's status.Read the evidence table

-
Self-reported Terminal-Bench 2.1 results: Ouroboros 86.74% with Claude Opus-5 after a disclosed reward-hack correction, ahead of Claude Code, Codex, Cursor, and Hermes on matched models
Terminal-Bench 2.1 · 86.74% corrected · self-reportedClosed submission (not merged) · Full run
+
Self-reported Terminal-Bench 2.1 results: Ouroboros 86.74% with Claude Opus-5 after a disclosed reward-hack correction, ahead of Claude Code, Codex, Cursor, and Hermes on matched models
Terminal-Bench 2.1 · 86.74% corrected · self-reportedSubmission · Full run
Self-reported OSWorld-Verified results: Ouroboros 90.69% with Claude Opus-5 and 83.27% with Claude Sonnet-4.6
OSWorld-Verified · 90.69% · self-reportedFull traces
Self-reported CL-Bench result: Ouroboros normalized reward 0.2301 with Claude Sonnet-4.6, rank 1
CL-Bench · 0.2301 · self-reported rank #1Open submission · Full traces
diff --git a/tests/test_public_site_metadata.py b/tests/test_public_site_metadata.py index 856393af6..9425be537 100644 --- a/tests/test_public_site_metadata.py +++ b/tests/test_public_site_metadata.py @@ -342,15 +342,16 @@ def test_readme_and_homepage_state_upstream_status_plainly(): for text in (readme, homepage): for line in text.splitlines(): if "terminal-bench-2-1/pull/" in line: - assert "closed" in line.lower() + if text == readme: + assert "Self-reported; community submissions closed" in line assert "submission open" not in line.lower() assert "open submission" not in line.lower() for root in (SITE, DOCS): benchmarks = (root / "benchmarks/index.html").read_text(encoding="utf-8") - assert "closed without merge on September 19, 2026" in benchmarks + assert "closed community submissions on September 19, 2026" in benchmarks for line in benchmarks.splitlines(): if "terminal-bench-2-1/pull/" in line: - assert "closed submission (not merged)" in line + assert "community submissions closed" in line def test_russian_talk_is_secondary_and_explicitly_labelled():
BenchmarkModelOuroborosComparisonStatus and evidence
Terminal-Bench 2.1Claude Opus-5 high86.74% after zeroing one disclosed reward-hack trial (raw: 86.97%)Claude Code + Fable 5: 83.8%Self-reported with a closed submission (not merged) and a public run.
Terminal-Bench 2.1Claude Opus-5 high86.74% after zeroing one disclosed reward-hack trial (raw: 86.97%)Claude Code + Fable 5: 83.8%Self-reported with a submission and a public run; community submissions closed.
Terminal-Bench 2.1Claude Opus-4.8 high80.22%Claude Code: 78.9%Self-reported with a public run.
Terminal-Bench 2.1GPT-5.584.3%Codex CLI: 83.1%Self-reported with a public run.
Terminal-Bench 2.1Grok-4.584.94%Cursor CLI: 79.3%. Hermes: 77.53%.Self-reported after a reward-hack audit, with a closed submission (not merged).
Terminal-Bench 2.1Grok-4.584.94%Cursor CLI: 79.3%. Hermes: 77.53%.Self-reported after a reward-hack audit, with a submission; community submissions closed.
OSWorld-VerifiedClaude Opus-590.69%Previous best on the public board: 90.19%Self-reported with public task evidence.
OSWorld-VerifiedClaude Sonnet-4.683.27%Pointer: 81.45%Self-reported with public task evidence.
CL-BenchClaude Sonnet-4.60.2301, rank 1Previous top: 0.1960Self-reported with an open submission and full traces.