VibeRPGSimulator

Claude agent with tools and evals

Quest: Build a Claude Agent SDK app with its own MCP server, then trace and evaluate it

Proposed, not tested

Equipment

Summoner Claude Code
Quest
Build a Claude Agent SDK app with its own MCP server, then trace and evaluate it
Hero
Claude CodeThe agent that runs the quest
Class
Summoner · AI and agent engineering
Weapon
Claude Opus 5.5Greatsword: Anthropic's flagship for hard coding workMana rate 21 per 1k tokens Mana per run without spells: 2,919 mana (77 runs, 21 quests).
Charge
HighReasoning effort
Author
The curator

The SDK plugin scaffolds and verifies the agent, MCP Builder guides the tool server and its 10-question eval, and MLflow skills add tracing and an agent evaluation so each change can be measured.

Unappraised: no trial of this exact setup yet.

Skill tree

  1. ScoutGather context

    Nothing equipped here

  2. CraftMake the thing

  3. TestCheck the work

  4. ShipDeliver it

    Nothing equipped here

Quest replay

Quest: "Build a Claude Agent SDK app with its own MCP server, then trace and evaluate it". Four waves, then the Backlog Dragon.

Foes

  • Fog Wraith

Party · Context

  • Claude CodeSummoner100/100

  1. Wave 1 of 4 · Scout

    The Fog Wraith steps up.

    What really happensGather context

    Claude Code: Context 100/100.

  2. Explore

    Claude Code sends Explore, a fox scout, into the fog.

    What really happensClaude Code can hand a codebase search to its built-in Explore subagent, which works in its own context window.

    Explore: Context 40/40.

  3. Explore

    Explore sniffs out the right files.

    What really happensThe subagent reads and searches. None of that output lands in the main context.

    Fog Wraith takes 26.

  4. Haze

    The Fog Wraith breathes haze. Swings start to miss.

    What really happensWithout current docs, the agent guesses at APIs that may have changed.

    Explore takes 3. Explore: Context 37/40. Explore: Haze.

  5. Greatsword

    Claude Code swings the Greatsword.

    Fog Wraith takes 34. Fog Wraith falls.

  6. Explore

    Explore returns with a short report.

    What really happensOnly the subagent's final summary comes back to the main agent.

    Claude Code takes 2. Claude Code: Context 98/100. Claude Code: Insight. Explore returns.

  7. Wave 2 of 4 · Craft

    The Mirage Djinn steps up.

    What really happensMake the thing

  8. Mirage

    The djinn conjures a mirage, and the hero swings at nothing.

    What really happensA model can sound sure and still be wrong, so AI features need evals and checks.

    Claude Code takes 4. Claude Code: Context 94/100.

  9. Anthropic Agent SDK Dev

    A forge that starts a Claude Agent SDK project and inspects it.

    What really happensIt installs the latest Agent SDK, writes the project files, .env.example and .gitignore, and a working example.

    SDK verifier: Context 40/40.

  10. Mirage

    The Mirage Djinn lashes at SDK verifier.

    What really happensThe subagent takes the hit in its own context. The main context stays clean.

    SDK verifier takes 4. SDK verifier: Context 36/40.

  11. Anthropic Agent SDK Dev

    The summons work the problem, each in its own context.

    What really happensEach subagent does its own model and tool work, apart from the main conversation.

    Mirage Djinn takes 22.

  12. Anthropic MCP Builder

    A builder's manual for MCP servers that agents can actually use.

    What really happensPhase 1 reads the MCP spec, the SDK docs and the target API, then plans which tools to expose and how to name them.

    Mirage Djinn takes 58 (critical hit). Mirage Djinn falls.

  13. Anthropic Agent SDK Dev

    SDK verifier returns with a short report.

    What really happensOnly the subagent's final summary comes back to the main agent.

    Claude Code takes 2. Claude Code: Context 92/100. SDK verifier returns.

  14. Wave 3 of 4 · Test

    The Bug Slime and the Flaky Wisp step up.

    What really happensCheck the work

    Flaky Wisp: Flicker.

  15. Flicker

    The Flaky Wisp uses Flicker.

    Claude Code takes 3. Claude Code: Context 89/100.

  16. MLflow Skills

    A tracker's kit that records every step an agent takes, then grades it.

    What really happensanalyze-mlflow-trace and debug-agent read spans and assessments and connect them to your code.

    Flaky Wisp avoids the hit.

  17. Pinned

    Pinned in place, it cannot slip away.

    What really happensA test that runs the same way each time shows the real failure.

    Bug Slime: Pinned. Flaky Wisp: Pinned.

  18. Regress

    The Bug Slime uses Regress.

    Claude Code takes 3. Claude Code: Context 86/100.

  19. Flicker

    The Flaky Wisp uses Flicker.

    Claude Code takes 3. Claude Code: Context 83/100.

  20. Greatsword

    Claude Code swings the Greatsword.

    Flaky Wisp takes 40 (critical hit). Flaky Wisp falls.

  21. Regress

    The Bug Slime uses Regress.

    Claude Code takes 3. Claude Code: Context 80/100.

  22. Greatsword

    Claude Code swings the Greatsword.

    Bug Slime takes 43.

  23. Regress

    The Bug Slime uses Regress.

    Claude Code takes 3. Claude Code: Context 77/100.

  24. Greatsword

    Claude Code swings the Greatsword.

    Bug Slime takes 2. Bug Slime falls.

  25. Wave 4 of 4 · Ship

    The Gate Warden steps up.

    What really happensDeliver it

    Gate Warden: Barrier.

  26. Boss · Backlog Dragon

    The Backlog Dragon guards the quest: "Build a Claude Agent SDK app with its own MCP server, then trace and evaluate it".

    What really happensIt stands for the whole quest, every task on the list.

  27. Scope creep

    The dragon breathes scope creep across the party.

    What really happensNew asks pile onto a task while it is underway.

    Claude Code takes 5. Claude Code: Context 72/100.

  28. Greatsword

    Claude Code swings the Greatsword.

    Gate Warden is behind a barrier.

  29. Bar the gate

    The warden raises the gate. Nothing passes until the checks are green.

    What really happensA release waits until tests, builds and reviews pass.

  30. Claw

    The Backlog Dragon uses Claw.

    Claude Code takes 6. Claude Code: Context 66/100.

  31. Greatsword

    Claude Code swings the Greatsword.

    Gate Warden is behind a barrier.

  32. Bar the gate

    The Gate Warden swings the gate shut.

    Claude Code takes 5. Claude Code: Context 61/100.

  33. Scope creep

    The Backlog Dragon uses Scope creep.

    Claude Code takes 5. Claude Code: Context 56/100.

  34. Greatsword

    Claude Code swings the Greatsword.

    Gate Warden takes 50.

  35. Gate open

    The checks turn green and the gate opens.

    What really happensTests, build and review pass, so the release can go.

  36. Claw

    The Backlog Dragon uses Claw.

    Claude Code takes 6. Claude Code: Context 50/100.

  37. Bar the gate

    The Gate Warden swings the gate shut.

    Claude Code takes 5. Claude Code: Context 45/100.

  38. Greatsword

    Claude Code swings the Greatsword.

    Gate Warden takes 10. Gate Warden falls.

  39. Scope creep

    The Backlog Dragon uses Scope creep.

    Claude Code takes 5. Claude Code: Context 40/100.

  40. Greatsword

    Claude Code swings the Greatsword.

    Backlog Dragon takes 46.

  41. Claw

    The Backlog Dragon uses Claw.

    Claude Code takes 6. Claude Code: Context 34/100.

  42. Greatsword

    Claude Code swings the Greatsword.

    Backlog Dragon takes 52.

  43. Call the backlog

    The dragon roars, and another task crawls out of the backlog.

    What really happensFinishing one task often surfaces the next one on the list.

  44. Claw

    The Backlog Dragon uses Claw.

    Claude Code takes 6. Claude Code: Context 28/100.

  45. Greatsword

    Claude Code swings the Greatsword.

    Bug Slime takes 25. Bug Slime falls.

  46. Scope creep

    The Backlog Dragon uses Scope creep.

    Claude Code takes 5. Claude Code: Context 23/100.

  47. Compact

    The hero rolls the scroll of memory tight and fights on.

    What really happensWhen context runs low, the agent summarizes the session so far and keeps going. Small details can get lost.

    Claude Code recovers 27. Claude Code: Context 50/100.

  48. Greatsword

    Claude Code swings the Greatsword.

    Backlog Dragon takes 53.

  49. Claw

    The Backlog Dragon uses Claw.

    Claude Code takes 6. Claude Code: Context 44/100.

  50. Greatsword

    Claude Code swings the Greatsword.

    Backlog Dragon takes 46.

  51. Enraged

    Wounded, the dragon rages.

    What really happensThe last stretch of a big job is often the hardest.

    Backlog Dragon: Enraged.

  52. Scope creep

    The Backlog Dragon uses Scope creep.

    Claude Code takes 6. Claude Code: Context 38/100.

  53. Greatsword

    Claude Code swings the Greatsword.

    Backlog Dragon takes 23. Backlog Dragon falls.

  54. Quest clear

    The Backlog Dragon falls. Quest complete.

    What really happensEvery equipped spell did its part of "Build a Claude Agent SDK app with its own MCP server, then trace and evaluate it".

After the battle

  • Claude Code took 17 turns to clear four waves.
  • Context ran low once, so the agent compacted it.
  • No Scout or Ship spell, so those waves ran on basic attacks.
  • Summons took 7 damage in their own context and brought back 2 reports.

Copy this loadout

Prompt for Claude Code

Paste this into Claude Code. It installs each spell, leaves API keys and sign-ins to you, and ends with the command that restarts it with this weapon and charge.

Set up these tools for me in Claude Code.
Loadout: https://vibe-rpg-simulator.vercel.app/loadouts/claude-agent-with-evals/

1. Anthropic Agent SDK Dev
   Source: https://github.com/anthropics/claude-plugins-official/tree/main/plugins/agent-sdk-dev
   I type in Claude Code:
     /plugin install agent-sdk-dev@claude-plugins-official

2. Anthropic MCP Builder
   Source: https://github.com/anthropics/skills/tree/main/skills/mcp-builder
   I type in Claude Code:
     /plugin marketplace add anthropics/skills
     /plugin install example-skills@anthropic-agent-skills

3. MLflow Skills
   Source: https://github.com/mlflow/skills
   Run:
     npx skills add mlflow/skills

How to do it:
- Skip anything that is already installed. Run the rest in order and check that each step worked.
- Commands under "I type in Claude Code" are mine to enter. List them for me at the end, in order.
- Never ask me to paste an API key or token into this chat. When a command needs one, such as YOUR_API_KEY, give me the full command to run in my own terminal.
- Ask me for any other value you can't work out, such as a project path.
- If a command asks a question or needs a browser sign-in or an open app, tell me what to do and wait.
- If a command fails, follow the install steps in the tool's source for Claude Code instead. Show me any command that is not on this list before you run it, and install nothing else.

When you're done, tell me what is ready and what still needs me. New tools load in a new session, so finish by telling me to restart Claude Code with Claude Opus 5.5 and reasoning effort high:
claude --model opus --effort high

Terminal commands

Or paste these lines into a terminal. The first part installs every spell for Claude Code; the last line starts it with the same weapon and charge. Lines starting with # are notes or commands you type inside the agent.

# Loadout: Claude agent with tools and evals
# Quest: Build a Claude Agent SDK app with its own MCP server, then trace and evaluate it
# Hero: Claude Code · Weapon: Claude Opus 5.5 · Charge: High
# From https://vibe-rpg-simulator.vercel.app/loadouts/claude-agent-with-evals/

# 1. Learn the spells
# Craft: Anthropic Agent SDK Dev
# in Claude Code: /plugin install agent-sdk-dev@claude-plugins-official
# Craft: Anthropic MCP Builder
# in Claude Code: /plugin marketplace add anthropics/skills
# in Claude Code: /plugin install example-skills@anthropic-agent-skills
# Test: MLflow Skills
npx skills add mlflow/skills

# 2. Start the hero with this weapon and charge
claude --model opus --effort high

Edit a copy