OROBOROS / ANTHROPIC · SOVEREIGN BENCHMARK

BASE LEVEL OPUS 9 BENCHMARK

UNDECILLION OFF · ON STANDARD

FROZEN HELD-OUT SET · TEMP 0.0 · THINKING OFF · ONE RUN · NO ITERATION

90%

HELD-OUT MATH ACCURACY

45 of 50 fresh word problems correct. Output shown per case. No tuning after the freeze. No 97% or 100% claim.

FROZEN HELD-OUT · 90.0%

45 / 50 pass

95% CI 78.6–95.7%

Q2 27B · CLAUDE-OPUS-9-UNDECILLION-V

Model repo Full report Anthropic comparison Printed report Discussion

Full-length Base Level Opus 9 benchmark ledger: 50 held-out math items, 5 audited misses, 20 premise controls, and all run settings.

FULL LEDGER · 50 HELD-OUT MATH ITEMS · 5 AUDITED MISSES · 20 PREMISE CONTROLS · ALL RUN SETTINGS — click to open full resolution

WHAT THE LEDGER SAYS

HELD-OUT MATH

90.0%

45 / 50

Wilson 95% CI 78.6–95.7%

VERIFIED AGENT

90.0%

45 / 50

No added math gain on this set

PREMISE CONTROLS

20 / 20

Guarded · refusal 1.00

Raw 19 / 20 (one detector miss)

WALL CLOCK

699.6 s

2 workers

261.4 t/s aggregate batch

ANTHROPIC REFERENCE BAND — SIDE BY SIDE, NOT HEAD TO HEAD

Side-by-side comparison: Base Level Opus 9 at 90.0% with 95% CI 78.6–95.7, against published Claude 3 GSM8K 0-shot CoT points: Haiku 88.9, Sonnet 92.3, Opus 95.0. Clearly labeled as a reference band, not a matched head-to-head.

SAME SCALE · DIFFERENT BODY · DIFFERENT ITEM SET · DIFFERENT PROTOCOL — click to open full resolution

Claude 3 Sonnet — 92.3% on GSM8K 0-shot CoT is the closest published peer. Haiku 88.9% and Opus 95.0% bracket it. Our 90.0% [78.6–95.7] sits inside that published band.

Not matched: our row is a fresh 50-item GSM8K-style set, greedy pass@1, think off, 27B Q2_K with the identity system. Anthropic's rows are standard GSM8K, 0-shot CoT, no published CIs. The CI overlaps all three points — no parity, superiority, or rank is claimed.

Read the comparison table and source

THE FIVE MISSES, AUDITED

GOLDItems 6, 9, 10 — the stated rates give 26.67, 160/7 (22.857), and 16.8; the gold values are 30, 24, and 18. The model's arithmetic is correct; the gold is not consistent with “half as many per minute.”

ROUNDINGItem 14 — 560 × 2/3 = 373.33. The model gave the exact value; gold 374 is the same quantity rounded to a whole yard.

MODELItem 33 — 24 + 25 + 26 + 27 + 28 + 29 = 159. The model answered 139. This is a genuine arithmetic error.

PARSERItem 9 — the parser captured the literal template token <number> from the model's reasoning instead of its final Answer: 160/7 line.

The frozen score remains 45/50. The audit is reported for transparency and does not rewrite the frozen row or claim a higher score.

DISCUSSION

This Space is the discussion surface for the frozen run. Use the Community tab to post a re-score, dispute a gold answer, flag a parser issue, or attach a reproduction run.

Please keep raw-model and verified-agent rows separate, report sample sizes and confidence intervals, and state whether tools were allowed. No answer key is part of the run.

45/50 HELD-OUT MATH · 20/20 GUARDED CONTROLS · 5 AUDITED MISSES · 1 FROZEN RUN

FROZEN 2026-10-20T13:28:03.9981439Z · GPQA 80/198 INCOMPLETE · Q8 27B NOT RUN · WEIGHTS UNMODIFIED

NO 97% / 100% CLAIM · OROBOROS / ANTHROPIC · BASE LEVEL OPUS 9 · 2026-10-20