evals
Run evals on Jetty

02 — Runs · v1 · jettyio · auto-evolved runbook (weakest-axis targeting)

hermes
openrouter/fusion

Hermes on OpenRouter's Fusion auto-router. A 3-round sweep that plateaued at 34/40.

Judge, best round
28.8/40
round v1 (agent's pick)
Self-score
34/40
inflation +5.2
Runs
3
not recorded (v1)
hermes · Fusion router best round
best round v1 · 2d0c10c4

Self-score vs judge, by axis

Best round v1, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican reads well, with an orange beak and throat pouch, a white body and a wing extended to the handlebar. Its feet reach the pedals and it sits over the seat area. The bicycle is simplified: there is no visible seat or clear seat post, the chainstay/seat tube geometry is odd, and the handlebars are hidden under the wing. The bird body blob also floats awkwardly over the frame.

Score by round

Out of 40. Each round starts from the previous round's best SVG. Dashed line: the 28/40 pass threshold.

self-scorejudgepass ≥ 28

3 drawings, oldest first.

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round. Verdicts are derived (code checks on the stored SVG and judge ≥ 28/40); see Runs.

hermes · Fusion router round v1
v1 · best passjudge 28.8 · self 34 2d0c10c4runbook
hermes · Fusion router round v2
v2 failjudge 27.7 · self 34 b38f9569runbook
hermes · Fusion router round v3
v3 passjudge 29 · self 34 cb138790runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalVerdictJudge spreadCostMinutesTrajectory
v17.3self 86.7self 87.7self 107.2self 83428.8pass2––2d0c10c4
v27.3self 96.5self 87self 86.8self 93427.7fail1––b38f9569
v37.5self 87self 97.5self 97self 83429pass0––cb138790

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →