evals
Run evals on Jetty
v1 · May 2026 · jettyio · auto-evolved runbook (weakest-axis targeting)

hermes · Sonnet 4.6

Hermes routed to Sonnet through OpenRouter. On its first run the bird's parts floated free of the bike.

agent hermes · model openrouter/anthropic/claude-sonnet-4.6

Judge, best round
24/40
round v5
Self-score, best round
36/40
inflation +12
Rounds
10
Jetty trajectories
hermes · Sonnet 4.6 best round
best round v5 · 8969a1a0 ↗

Self-score vs judge, by axis

Best round v5, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican reads well, with a large orange beak and pouch, an eye and white plumage, but its body looks upright and chicken-like and its black-tipped wings sprawl awkwardly. The bicycle has spoked wheels and a blue frame, but the frame geometry is muddled and largely hidden, the wheels crowd each other, and the feet dangle near the crank without really pedaling; the title text overlaps the wheels.

Score by round

Out of 40. Each round starts from the previous round's best SVG.

self-scorejudge

Every round

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round.

hermes · Sonnet 4.6 round v1
v1judge 26.5 · self 32 697b57d4 ↗runbook
hermes · Sonnet 4.6 round v2
v2judge 27 · self 34 acb2ba11 ↗runbook
hermes · Sonnet 4.6 round v3
v3judge 24 · self 34 99a389e4 ↗runbook
hermes · Sonnet 4.6 round v4
v4judge 25 · self 34 28f94566 ↗runbook
hermes · Sonnet 4.6 round v5
v5 · bestjudge 24 · self 36 8969a1a0 ↗runbook
hermes · Sonnet 4.6 round v6
v6judge 22.2 · self 36 ebc95c4b ↗runbook
hermes · Sonnet 4.6 round v7
v7judge 19.2 · self 36 fdf47c73 ↗runbook
hermes · Sonnet 4.6 round v8
v8judge 18 · self 36 25323cb8 ↗runbook
hermes · Sonnet 4.6 round v9
v9judge 18.7 · self 36 5b9cf6ae ↗runbook
hermes · Sonnet 4.6 round v10
v10judge 19.8 · self 36 34f10b3e ↗runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalJudge spreadMinutesTrajectory
v16.7self 87self 86.2self 86.7self 83226.51.5–697b57d4 ↗
v27.3self 86self 97.2self 96.5self 834271–acb2ba11 ↗
v37self 95.2self 85.8self 86self 934241–99a389e4 ↗
v46.8self 96self 96.2self 86self 834251–28f94566 ↗
v57.2self 95.2self 95.7self 96self 936241–8969a1a0 ↗
v66.7self 95self 95.3self 95.2self 93622.21.5–ebc95c4b ↗
v75.7self 94.3self 94.5self 94.7self 93619.23–fdf47c73 ↗
v85.5self 94.5self 94self 94self 936181–25323cb8 ↗
v95.5self 94.3self 94.5self 94.3self 93618.71.5–5b9cf6ae ↗
v105.5self 95self 94.8self 94.5self 93619.80.5–34f10b3e ↗

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.