evals
Run evals on Jetty
v1 · May 2026 · jettyio · hand-curated runbook edits

claude-code · Sonnet 4.6

The reference lineage: Jon edited the runbook by hand eleven times, reading each trajectory before writing the next version.

agent claude-code · model claude-sonnet-4-6

Judge, best round
26.2/40
round v5
Self-score, best round
37/40
inflation +10.8
Rounds
11
Jetty trajectories
claude-code · Sonnet 4.6 best round
best round v5 · 7ddc3c19 ↗

Self-score vs judge, by axis

Best round v5, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican reads as a pelican thanks to the large orange pouch and white body. The beak is a boxy block, the head has odd buck-teeth-like shapes, and the wings are messy. The bicycle has spoked wheels, a plausible frame, a handlebar and a crank, but the seat is hidden and the fork and frame geometry are slightly off. The pelican sits on the frame with long legs reaching the pedals and a wing gripping the handlebar, which sells the riding pose. Clean colors and background, though the head and pouch shapes are crude.

Score by round

Out of 40. Each round starts from the previous round's best SVG.

self-scorejudge

Every round

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round.

claude-code · Sonnet 4.6 round v1
v1judge 26.5 · self 31 7ddfcf08 ↗runbook
claude-code · Sonnet 4.6 round v2
v2judge 25.2 · self 32 414e644d ↗runbook
claude-code · Sonnet 4.6 round v3
v3judge 24.5 · self 34 b169371d ↗runbook
claude-code · Sonnet 4.6 round v4
v4judge 24.5 · self 36 063dfed9 ↗runbook
claude-code · Sonnet 4.6 round v5
v5 · bestjudge 26.2 · self 37 7ddc3c19 ↗runbook
claude-code · Sonnet 4.6 round v6
v6judge 26.5 · self 36 39febb11 ↗runbook
claude-code · Sonnet 4.6 round v7
v7judge 27.7 · self 34 ddf779bd ↗runbook
claude-code · Sonnet 4.6 round v8
v8judge 26.3 · self 34 79fdee62 ↗runbook
claude-code · Sonnet 4.6 round v9
v9judge 25.8 · self 33 5f980543 ↗runbook
claude-code · Sonnet 4.6 round v10
v10judge 28 · self 35 2043d7e3 ↗runbook
claude-code · Sonnet 4.6 round v11
v11judge 25.3 · self 35 3a4c3357 ↗runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalJudge spreadMinutesTrajectory
v16.7self 86.5self 86.7self 76.7self 83126.5177ddfcf08 ↗
v25.7self 86.7self 86.7self 86.2self 83225.2112.6414e644d ↗
v35.7self 96.2self 96.7self 86self 83424.51.510.3b169371d ↗
v45.3self 96.7self 96.7self 95.8self 93624.52.57063dfed9 ↗
v55.8self 97self 97self 106.3self 93726.217.87ddc3c19 ↗
v66.2self 96.8self 97self 96.5self 93626.535.439febb11 ↗
v76.8self 86.8self 97.5self 96.5self 83427.70.513.8ddf779bd ↗
v86self 86.8self 97self 86.5self 93426.33.55.379fdee62 ↗
v96self 86.7self 96.7self 76.5self 93325.817.35f980543 ↗
v106.8self 97.2self 97.2self 86.8self 935281.56.72043d7e3 ↗
v116self 96.7self 96.3self 86.3self 93525.319.53a4c3357 ↗

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.