evals
Run evals on Jetty

02 — Runs · v1 · jettyio · hand-curated runbook edits

claude-code
claude-sonnet-4-6

The reference lineage: Jon edited the runbook by hand eleven times, reading each trajectory before writing the next version.

Judge, best round
26.2/40
round v5 (agent's pick)
Self-score
37/40
inflation +10.8
Runs
11
not recorded (v1)
claude-code · Sonnet 4.6 best round
best round v5 · 7ddc3c19

Self-score vs judge, by axis

Best round v5, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican reads as a pelican thanks to the large orange pouch and white body. The beak is a boxy block, the head has odd buck-teeth-like shapes, and the wings are messy. The bicycle has spoked wheels, a plausible frame, a handlebar and a crank, but the seat is hidden and the fork and frame geometry are slightly off. The pelican sits on the frame with long legs reaching the pedals and a wing gripping the handlebar, which sells the riding pose. Clean colors and background, though the head and pouch shapes are crude.

Score by round

Out of 40. Each round starts from the previous round's best SVG. Dashed line: the 28/40 pass threshold.

self-scorejudgepass ≥ 28

11 drawings, oldest first.

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round. Verdicts are derived (code checks on the stored SVG and judge ≥ 28/40); see Runs.

claude-code · Sonnet 4.6 round v1
v1 failjudge 26.5 · self 31 7ddfcf08runbook
claude-code · Sonnet 4.6 round v2
v2 failjudge 25.2 · self 32 414e644drunbook
claude-code · Sonnet 4.6 round v3
v3 failjudge 24.5 · self 34 b169371drunbook
claude-code · Sonnet 4.6 round v4
v4 failjudge 24.5 · self 36 063dfed9runbook
claude-code · Sonnet 4.6 round v5
v5 · best failjudge 26.2 · self 37 7ddc3c19runbook
claude-code · Sonnet 4.6 round v6
v6 failjudge 26.5 · self 36 39febb11runbook
claude-code · Sonnet 4.6 round v7
v7 failjudge 27.7 · self 34 ddf779bdrunbook
claude-code · Sonnet 4.6 round v8
v8 failjudge 26.3 · self 34 79fdee62runbook
claude-code · Sonnet 4.6 round v9
v9 failjudge 25.8 · self 33 5f980543runbook
claude-code · Sonnet 4.6 round v10
v10 passjudge 28 · self 35 2043d7e3runbook
claude-code · Sonnet 4.6 round v11
v11 failjudge 25.3 · self 35 3a4c3357runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalVerdictJudge spreadCostMinutesTrajectory
v16.7self 86.5self 86.7self 76.7self 83126.5fail1–77ddfcf08
v25.7self 86.7self 86.7self 86.2self 83225.2fail1–12.6414e644d
v35.7self 96.2self 96.7self 86self 83424.5fail1.5–10.3b169371d
v45.3self 96.7self 96.7self 95.8self 93624.5fail2.5–7063dfed9
v55.8self 97self 97self 106.3self 93726.2fail1–7.87ddc3c19
v66.2self 96.8self 97self 96.5self 93626.5fail3–5.439febb11
v76.8self 86.8self 97.5self 96.5self 83427.7fail0.5–13.8ddf779bd
v86self 86.8self 97self 86.5self 93426.3fail3.5–5.379fdee62
v96self 86.7self 96.7self 76.5self 93325.8fail1–7.35f980543
v106.8self 97.2self 97.2self 86.8self 93528pass1.5–6.72043d7e3
v116self 96.7self 96.3self 86.3self 93525.3fail1–9.53a4c3357

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →