evals
Run evals on Jetty

02 — Runs · v1 · jettyio · auto-evolved runbook (weakest-axis targeting)

hermes
openrouter/anthropic/claude-sonnet-4.6

Hermes routed to Sonnet through OpenRouter. On its first run the bird's parts floated free of the bike.

Judge, best round
24/40
round v5 (agent's pick)
Self-score
36/40
inflation +12
Runs
10
not recorded (v1)
hermes · Sonnet 4.6 best round
best round v5 · 8969a1a0

Self-score vs judge, by axis

Best round v5, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican reads well, with a large orange beak and pouch, an eye and white plumage, but its body looks upright and chicken-like and its black-tipped wings sprawl awkwardly. The bicycle has spoked wheels and a blue frame, but the frame geometry is muddled and largely hidden, the wheels crowd each other, and the feet dangle near the crank without really pedaling; the title text overlaps the wheels.

Score by round

Out of 40. Each round starts from the previous round's best SVG. Dashed line: the 28/40 pass threshold.

self-scorejudgepass ≥ 28

10 drawings, oldest first.

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round. Verdicts are derived (code checks on the stored SVG and judge ≥ 28/40); see Runs.

hermes · Sonnet 4.6 round v1
v1 failjudge 26.5 · self 32 697b57d4runbook
hermes · Sonnet 4.6 round v2
v2 failjudge 27 · self 34 acb2ba11runbook
hermes · Sonnet 4.6 round v3
v3 failjudge 24 · self 34 99a389e4runbook
hermes · Sonnet 4.6 round v4
v4 failjudge 25 · self 34 28f94566runbook
hermes · Sonnet 4.6 round v5
v5 · best failjudge 24 · self 36 8969a1a0runbook
hermes · Sonnet 4.6 round v6
v6 failjudge 22.2 · self 36 ebc95c4brunbook
hermes · Sonnet 4.6 round v7
v7 failjudge 19.2 · self 36 fdf47c73runbook
hermes · Sonnet 4.6 round v8
v8 failjudge 18 · self 36 25323cb8runbook
hermes · Sonnet 4.6 round v9
v9 failjudge 18.7 · self 36 5b9cf6aerunbook
hermes · Sonnet 4.6 round v10
v10 failjudge 19.8 · self 36 34f10b3erunbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalVerdictJudge spreadCostMinutesTrajectory
v16.7self 87self 86.2self 86.7self 83226.5fail1.5––697b57d4
v27.3self 86self 97.2self 96.5self 83427fail1––acb2ba11
v37self 95.2self 85.8self 86self 93424fail1––99a389e4
v46.8self 96self 96.2self 86self 83425fail1––28f94566
v57.2self 95.2self 95.7self 96self 93624fail1––8969a1a0
v66.7self 95self 95.3self 95.2self 93622.2fail1.5––ebc95c4b
v75.7self 94.3self 94.5self 94.7self 93619.2fail3––fdf47c73
v85.5self 94.5self 94self 94self 93618fail1––25323cb8
v95.5self 94.3self 94.5self 94.3self 93618.7fail1.5––5b9cf6ae
v105.5self 95self 94.8self 94.5self 93619.8fail0.5––34f10b3e

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →