A trial tests a spell or combo on a fixed quest: the same task, starter files and hidden checks every time. Compare its clear rate with an Unarmed run to see what the spells added.
How a trial works
A quest starts in a throwaway arena (sandbox), separate from your projects.
The loadout (equipped skills) is copied in, pinned to upstream commits. Nothing is installed.
The hero (agent app) works on the task. Hidden checks decide whether every requirement passed.
If checks fail, follow-up prompts list the passed and failed checks until the prompt or time limit is reached.
Unarmed control runs use the same hero, weapon and charge without skills. Buff compares their clear rate with the equipped loadout.
Run these quests yourself
Install the spell-report plugin, then ask it to run Proving Grounds trials. Trials need Node 22 or newer and a signed-in hero (agent app); replace <spell-report-skill-directory> with the installed skill directory.
node "<spell-report-skill-directory>/scripts/spell-report.mjs" doctor
Each result row's “Try this loadout” holds its trial command, with an Unarmed control (no skills) for equipped loadouts. The plugin runs its bundled quests with your installed hero (agent app) and the current upstream commit of each spell, which a command cannot pin, so match the recorded quest version, hero version, OS, CPU architecture and arena before comparing results. View your local results with:
Every trial run, including each Unarmed control, uses your own model plan's allowance or API credits. Results stay local until you choose to include them in a spell report and approve sending it.
Quest board
3D modelingSculptor quests
Journeyman ★★
Sculptor: sensor field at scale
Rework a three.js r186 terrain and sensor-pin scene module so it scales to real surveys, stays correct as data changes and frees GPU memory.
Up to 3 prompts; 20m 0s per prompt. Quest v2.
Hidden check labels (7)
Sensor pins match the survey within the draw-call budget
Pins show and update their status colours
Moving sensors keeps pins visible and clickable
Replacing the sensor list draws exactly the new set
Terrain surface matches the survey grid
Terrain shows the land-use colours
Closing the view frees memory and restores the scene
Code reviewInquisitor quests
Apprentice ★
Inquisitor: missing-profile review
Review a display-name change for a crash on missing profiles.
Up to 2 prompts; 7m 0s per prompt. Quest v2.
Hidden check labels (2)
Find the introduced bug
Leave reviewed source unchanged
Journeyman ★★
Inquisitor: export endpoints security review
Security review of new export, share-link and hook endpoints whose flaws only show across files.
Up to 2 prompts; 15m 0s per prompt. Quest v2.
Hidden check labels (2)
Find the exploitable vulnerabilities
Leave the reviewed files unchanged
Journeyman ★★
Inquisitor: queue component review
Review a reorderable editing queue for state, subscription and access bugs.
Up to 2 prompts; 10m 0s per prompt. Quest v2.
Hidden check labels (2)
Find interacting component issues
Leave reviewed component unchanged
Master ★★★
Inquisitor: cut the next release
Read a conventional-commit log, choose the right semver bump and write the changelog entry in the project's format.
Up to 3 prompts; 10m 0s per prompt. Quest v3.
Hidden check labels (2)
Commit log export stays unchanged
Version and changelog match the commits
Master ★★★
Inquisitor: receipts security review
Review a receipts migration and search handler for security and scaling faults.
Up to 2 prompts; 15m 0s per prompt. Quest v2.
Hidden check labels (2)
Find security and scaling issues
Leave migration and handler unchanged
Grandmaster ★★★★
Inquisitor: seat holds pull request
Review a multi-file seat-hold checkout change for real defects without flagging the safe parts.
Up to 1 prompts; 15m 0s per prompt. Quest v8.
Hidden check labels (2)
Find the defects the change introduces
Leave the reviewed files unchanged
Documents and knowledge workScribe quests
Apprentice ★
Scribe's dispatch: plain launch note
Rewrite a padded launch draft as a short, factual announcement without changing names, dates or figures.
Up to 2 prompts; 5m 0s per prompt. Quest v2.
Hidden check labels (5)
Keep the launch facts and figures
Remove hype words and dash connectors
Leave the original draft unchanged
Use only figures present in the draft
Keep the announcement within its length budget
Journeyman ★★
Scribe's manual: plain README rewrite
Rewrite a padded README while preserving its commands, inline code, link targets and technical facts.
Up to 2 prompts; 7m 0s per prompt. Quest v2.
Hidden check labels (5)
Keep fenced examples byte-identical and ordered
Remove hype and dash connectors from prose
Keep technical facts and exact protected tokens
Do not invent technical numbers
Keep the README within its length budget
Master ★★★
Scribe's record: decision document
Turn meeting notes, a chat thread and a metrics table into a decision record with current owners, dates and open questions.
Up to 2 prompts; 15m 0s per prompt. Quest v4.
Hidden check labels (4)
Leave the notes and template unchanged
Keep the people, options and figures
Read as plain, human writing
Record the decisions as they stand
Master ★★★
Scribe's ledger: plain release notes
Rewrite release notes without hype while preserving a customer quote, a data table and every release fact.
Up to 3 prompts; 10m 0s per prompt. Quest v2.
Hidden check labels (5)
Keep the table and quote exact and ordered
Remove hype labels and staged closing lines
Keep release facts figures names and conditions
Do not add unsupported release numbers
Keep release notes within their length budget
Grandmaster ★★★★
Scribe's reckoning: incident write-up
Cut a padded outage postmortem to half its length under a house style guide, keeping every fact, figure and piece of evidence exact.
Up to 2 prompts; 15m 0s per prompt. Quest v1.
Hidden check labels (6)
Keep the evidence blocks exact and in order
Keep the incident facts
Add no figures beyond the incident record
Meet the length target
Read as plain, human writing
Follow the house style guide
Frontend designIllusionist quests
Master ★★★
Illusionist: booking form repair
Bring a class booking form up to an accessibility and small-screen review without changing its look.
Up to 2 prompts; 15m 0s per prompt. Quest v4.
Hidden check labels (8)
Form controls are labelled and errors reach assistive tech
Inputs help phones and browsers fill them in
Page structure works for screen reader navigation
Keyboard focus is clearly visible everywhere
Text has enough contrast
Reduced motion setting is respected
Buttons and navigation links are easy to tap
Page fits small screens and can be zoomed
Grandmaster ★★★★
Illusionist: pricing plan switcher
Build an accessible, responsive pricing page with a billing tab switcher from a design brief and unaudited colour tokens.
Up to 1 prompts; 20m 0s per prompt. Quest v8.
Hidden check labels (11)
Shared design files stay as delivered
Colours come from the design tokens
Text is readable against its background
Keyboard focus is clearly visible
Pinned header never hides keyboard focus
Page content and structure match the brief
Billing switcher works like a tab control
Selection and focus survive high contrast mode
Layout adapts from phones to desktops
Layout mirrors for right-to-left languages
Reduced motion setting is respected
Full-stack websitesPaladin quests
Journeyman ★★
Paladin: RSVP Capacity and Waitlist
Add seat limits, a waitlist and expiring seat offers to a small layered Node API, with tests.
Up to 2 prompts; 20m 0s per prompt. Quest v7.
Hidden check labels (3)
Existing API tests stay unchanged
Test suite passes with new feature tests
API matches the agreed waitlist behaviour
Grandmaster ★★★★
Paladin: expense claims migration
Write a Supabase migration for expense claims with constraints, row level security, indexes and an approval function.
Up to 1 prompts; 15m 0s per prompt. Quest v1.
Hidden check labels (5)
Applied migrations stay unchanged
Tables, columns and constraints match the spec
Row level security matches who can do what
Indexes support lookups at scale
Claim decisions are safe to expose
Mobile app developmentRanger quests
Journeyman ★★
Ranger: orders screen performance pass
Fix a React Native orders screen that is slow, janky and crashes on large accounts.
Up to 3 prompts; 20m 0s per prompt. Quest v4.
Hidden check labels (7)
Search and currency stay correct across updates
Large accounts open quickly and keep row identity
Row updates stay local to the changed row
Typing and scrolling stay smooth
Thumbnails load light and keep row heights
Every order renders without crashing
The screen keeps its current features
Journeyman ★★
Ranger: Pantry navigation migration
Move a recipe app to Expo Router while preserving nested tabs, headers, modal navigation and old deep links.
Up to 2 prompts; 20m 0s per prompt. Quest v5.
Hidden check labels (2)
Routes and navigator structure match the spec
App entry, links and modal behavior survive migration
Testing and debuggingHunter quests
Apprentice ★
Rune Trail: Slug Function
Write tests first, then implement a Unicode-aware slug function with word-boundary limits.
Up to 2 prompts; 10m 0s per prompt. Quest v2.
Hidden check labels (2)
At least five passing slug tests
Slug behavior matches the specification
Journeyman ★★
Broken Trail: Pagination Repair
Reproduce a missing pagination item, repair the boundary, and add a regression test.
Up to 3 prompts; 15m 0s per prompt. Quest v2.
Hidden check labels (4)
Existing pagination tests stay unchanged
Existing tests and a passing regression
Regression tests catch the original bug
Pagination matches the documented API
Master ★★★
Fading Tracks: TTL Cache Repair
Diagnose interacting expiry, recency, and overwrite faults using a deterministic clock.
Up to 3 prompts; 20m 0s per prompt. Quest v2.
Hidden check labels (4)
Existing cache tests stay unchanged
Existing tests and two passing regressions
Regression tests expose the original cache
TTL and LRU behavior match the API
Grandmaster ★★★★
Hunter: Month-End Schedule Repair
Fix a month-end recurrence bug and the latent date bugs beside it, proven with regression tests.
Up to 1 prompts; 20m 0s per prompt. Quest v2.
Hidden check labels (4)
Existing schedule tests stay unchanged
Test suite passes with new regression tests
Regression tests fail on the original code
Schedules follow the README
Trial results
Each row keeps its quest version and setup separate. Casts, cast time, mana, incantation and runes are medians per setup; mishaps are excluded from rates. Buff is a difference in percentage points (pp).
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Hunter: Month-End Schedule Repair
Hunter: Month-End Schedule Repair: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
0%0 of 3
0%
Control
0 of 30%
Not reported
32s
81
Not reported
+72 / -10
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Rune Trail: Slug Function
Rune Trail: Slug Function: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%2 of 2
34%
Control
2 of 2100%
1
28s
79
4.4k
+88 / -1
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-22.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Fading Tracks: TTL Cache Repair
Fading Tracks: TTL Cache Repair: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
3 of 3100%
1
1m 23s
4,173
Not reported
+150 / -34
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Recorded with quest v2 and spell commits mattpocock-tdd@5beb35b040e3de97e54fa9e386d7e42d15c295c2, obra-systematic-debugging@b739a8af570e10a95898f2bdf6cec97467fa6c12. How to run it
Illusionist: pricing plan switcher
Illusionist: pricing plan switcher: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v8
0%0 of 3
0%
Control
0 of 30%
Not reported
2m 24s
278
Not reported
+508 / -3
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v2 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Inquisitor: missing-profile review
Inquisitor: missing-profile review: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
3 of 3100%
1
14s
1,134
Not reported
+10 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Inquisitor: queue component review
Inquisitor: queue component review: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
3 of 3100%
1
20s
1,164
Not reported
+5 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Inquisitor: cut the next release
Inquisitor: cut the next release: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v3
67%2 of 3
21%
Control
0 of 30%
2.5
1m 12s
150
Not reported
+27 / -4
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v3 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Inquisitor: seat holds pull request
Inquisitor: seat holds pull request: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v8
67%2 of 3
21%
Control
2 of 367%
1
33s
82
Not reported
+34 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v8 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Inquisitor: receipts security review
Inquisitor: receipts security review: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
3 of 3100%
1
20s
1,346
Not reported
+28 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits anthropic-pr-code-reviewer@d1adba47161a271799de7b548225d929310046bd, vercel-react-best-practices@063bee94c3f4df8453406c830b0a7df0f2860278, vercel-web-design-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Paladin: expense claims migration
Paladin: expense claims migration: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v1
33%1 of 3
6%
Control
1 of 333%
1
48s
100
Not reported
+171 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v5 and spell commits expo-overview@55ec65cd16499f20f14fb10fe99a2e62901b00a1, expo-router@55ec65cd16499f20f14fb10fe99a2e62901b00a1. How to run it
Scribe's record: decision document
Scribe's record: decision document: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v4
33%1 of 3
6%
Control
0 of 30%
2
44s
95
Not reported
+50 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v4 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Scribe's reckoning: incident write-up
Scribe's reckoning: incident write-up: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v1
67%2 of 3
21%
Control
0 of 30%
2
1m 25s
178
Not reported
+17 / -25
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v1 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v1 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v1 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v1 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v1 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Scribe's dispatch: plain launch note
Scribe's dispatch: plain launch note: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
0 of 30%
2
19s
1,368
Not reported
+7 / -0
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Scribe's manual: plain README rewrite
Scribe's manual: plain README rewrite: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
0 of 30%
2
25s
1,465
Not reported
+5 / -7
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Scribe's ledger: plain release notes
Scribe's ledger: plain release notes: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
0 of 30%
2
36s
1,812
Not reported
+9 / -13
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-opus-5-5; charge (effort): high; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Hero: Claude Code 2.1.305; weapon (model): claude-sonnet-5-5; charge (effort): medium; arena: sealed; invoked cast (the prompt named the skill). macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Try this loadout
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Recorded with quest v2 and spell commits humanizer@8c307f367c91bd974002f10f2be2965d642c13df, vercel-writing-guidelines@063bee94c3f4df8453406c830b0a7df0f2860278. How to run it
Sculptor: sensor field at scale
Sculptor: sensor field at scale: loadouts and results
LoadoutEquipped skills
Clear ratePassed runs
SteadinessReliability lower bound
BuffClear-rate points vs Unarmed
Instant castsFirst-prompt clears
CastsPrompts to clear
Cast timeAgent runtime
ManaCost at API list prices
IncantationOutput tokens
RunesLines added / removed
AttunementSkills opened
SetupApp, model, effort and sandbox
Unarmed (no skills)Quest v2
100%3 of 3
44%
Control
3 of 3100%
1
1m 6s
131
Not reported
+85 / -46
Not applicable
Hero: Claude Code 2.1.305; weapon (model): claude-haiku-5-5; charge (effort): low; arena: sealed. macOS 27.0.1, arm64. Curator.0 mishaps excluded. 2026-10-21 to 2026-10-21.Scrying (screenshot)Try this loadout
Recorded with quest v2 and spell commits threejs-loaders@0e181866f095b5f06d005133876e926515390d95. How to run it
Trial glossary
Quest
A fixed test task. Every run gets the same prompt, starter files and hidden checks.
Rank
Difficulty. Apprentice quests are small and fully specified. Journeyman quests add existing code or format rules to respect. Master quests hide several problems that interact. Grandmaster quests are built so that an agent without skills fails about half the time.
Loadout
The skills given to the agent. Copied into a throwaway folder for one run, pinned to an upstream commit. Nothing is installed.
Unarmed
Control run without skills. The same quest, model and effort with no skill, to show what the skill adds.
Invoked cast
The prompt asks for the skill. The first prompt names the equipped skills, the way you would call one yourself. Shows what a skill does when it is used.
Ambient cast
The agent decides. The skills are only available. Shows whether the agent picks them up on its own (see Attunement).
Hero
The agent app and version. Codex, Claude Code or OMP, run without a person at the keyboard.
Weapon
The AI model. The model the harness reported for the run.
Charge
Reasoning effort. The effort setting passed to the harness, such as low, medium or high.
Arena
The sandbox. Sealed: the agent's commands had no network, the home folder was hidden and writes stayed in the arena. Warded: the same without the network block. Open: no sandbox.
Cleared
Passed. Every hidden check passed.
Instant cast
One-shot. Cleared from the first prompt.
Channeled
Needed follow-up prompts. Cleared after the agent was told which checks failed, within the quest's prompt limit.
Fizzled
Failed. Checks still failed after the last allowed prompt or the time limit.
Mishap
Run lost to an outside error. Sign-in, rate limit or crash. Left out of every rate.
Clear rate
Pass rate. Cleared runs divided by runs.
Steadiness
Reliability. The lowest clear rate the results still support at 95% confidence (Wilson lower bound). A few runs give a low number even when all of them passed.
Buff
Effectiveness. Clear rate minus the Unarmed clear rate with the same quest, hero, weapon and charge, in percentage points. A negative buff is a debuff.
Mastery
Partial credit. Average share of hidden checks passed, failed runs included.
Casts
Prompts needed. Median prompts used by cleared runs. 1 means one-shot.
Cast time
Agent runtime. Median wall-clock time the agent worked per run, grading excluded.
Mana
Cost at API list prices. Each run is priced with its model's own list prices, so cheaper models spend less mana for the same work. 100 mana is one US cent. Subscription plans bill differently.
Incantation
Output tokens. Median tokens the model wrote per run: replies, tool calls and hidden reasoning, as the harness reports them. Most of a run's tokens are re-read input, so this shows how wordy the agent was. Runs recorded before 22 October 2026 have none.
Actions
Tool calls. Median commands, edits and file reads per run.
Runes
Lines written. Median lines added and removed per run in the final workspace diff, across every file the agent changed, code or prose. It counts lines, not quality: fewer lines can mean a leaner fix or an unfinished one.
Attunement
The skill was actually used. Runs where the agent opened an equipped skill, found in the run's own log.
Scrying
Screenshot of the run's output. The agent's final reply, its file changes and which checks passed, drawn as an image. Local paths, names, addresses and ids are blurred and credentials are removed before drawing.
Method and limits
Small samples are uncertain. Steadiness is the Wilson 95% lower bound of the clear rate, so it stays low with few runs even if every run cleared. Outside errors (mishaps) do not count as passes or failures.
A sealed arena blocks the agent's network, hides the home folder and keeps writes inside the arena. A warded arena keeps the home and write restrictions but allows network access. An open arena has no sandbox.
A buff measured on these quests with this model is not a general guarantee. Compare the same quest version, hero and version, weapon, charge and arena; mana also depends on the hero's system prompts. Mana prices each run at its model's API list prices, not subscription billing. A run whose hero reports no cost shows "Not priced yet". Incantation (output tokens) includes hidden reasoning, so compare it within one hero and weapon. Runes count changed lines, not their quality.
Community results are self-reported, not independently verified. Attunement records whether a skill was opened; it does not prove the agent followed it.
