evals
Run evals on Jetty

02 — Runs · v2 · jettygrowthteam · auto-evolved runbook (weakest-axis targeting)

hermes
typesafe/jev-router

hermes driving typesafe/jev-router through OpenRouter, 5 hill-climb rounds from the v1 runbook.

Judge, best round
31.7/40
round v4 (agent's pick)
Self-score
37/40
inflation +5.3
Runs
5
not metered
hermes · Jev Router best round
best round v4 · 3ba61c00

Self-score vs judge, by axis

Best round v4, 0–10 per axis

agent self-scoreexternal judge

Judge: A clearly recognizable pelican with a large orange pouch bill sits on the saddle, feet on the pedals and wing tip on the handlebars. The bicycle is well formed, with spoked wheels, a plausible frame, and a chain. Minor defects are the spindly, awkwardly bent legs, the oversized dark saddle, and a somewhat crude tail and wing shape.

Score by round

Out of 40. Each round starts from the previous round's best SVG. Dashed line: the 28/40 pass threshold.

self-scorejudgepass ≥ 28

5 drawings, oldest first.

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round. Verdicts are derived (code checks on the stored SVG and judge ≥ 28/40); see Runs.

hermes · Jev Router round v1
v1 passjudge 29.8 · self 36 1a125d51runbook
hermes · Jev Router round v2
v2 passjudge 31.5 · self 35 7a63ffb0runbook
hermes · Jev Router round v3
v3 passjudge 30.7 · self 34.5 ded99127runbook
hermes · Jev Router round v4
v4 · best passjudge 31.7 · self 37 3ba61c00runbook
hermes · Jev Router round v5
v5 passjudge 30.8 · self 36 ef67915drunbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalVerdictJudge spreadCostMinutesTrajectory
v17.3self 97self 98self 97.5self 93629.8pass0.5not metered2.21a125d51
v28self 97.3self 98.5self 97.7self 83531.5pass1not metered1.67a63ffb0
v37.7self 97.5self 98.2self 87.3self 8.534.530.7pass2.5not metered2.6ded99127
v48self 97.7self 98.3self 107.7self 93731.7pass0.5not metered23ba61c00
v57self 98self 98.3self 97.5self 93630.8pass0.5not metered3.1ef67915d

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →