evals
Run evals on Jetty

02 — Run · Round 2 · Round robin · r2-rr-jev-sonnet5-g2

Claude Sonnet 5 vs Jev Router

Jev Router destroyed the enemy base after 5.0 game-minutes · 7.2 min wall clock

LOSS england · west
Claude Sonnet 5
anthropic/claude-sonnet-5
Value destroyed$600Value lost$12,800 Units killed / lost2 / 4Buildings killed / lost0 / 11 Peak army$1,400Orders issued53 Decision turns (failed)38 (0)Mean latency4.4sModel cost$0.224
WIN germany · east
Jev Router
typesafe/jev-router
Value destroyed$12,800Value lost$600 Units killed / lost4 / 2Buildings killed / lost11 / 0 Peak army$2,600Orders issued84 Decision turns (failed)38 (0)Mean latency4.2sModel cost$0.065

Evaluation

passchecks 10/10 verdict computed from the gating checks

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, content, key)
passsteplaunchLaunch run-match as a detached process
passstepsuperviseSupervise in the foreground until run-match exits
passsteprenderRender the 1080p recording
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checkgame-has-winnerjev won: base destroyed
passcode_checkdecisions-loggedturns per slot: {'Multi0': 38, 'Multi1': 38}
passcode_checkllm-turns-validfailed/total per slot: {'Multi0': '0/38', 'Multi1': '0/38'}
passcode_checkvideo-decodesduration 51.8s
passcode_checkvideo-completevideo ends at tick 7466 of 7466
passcode_checklimits-respectedended after 7.2 wall-minutes (budget 30.0 min): base destroyed
passchecklistnotes-explain-winsummary.md Notes section cites both models final thoughts and explains Jevs infantry sweep victory
passchecklistplayers-match-requestresult.json shows Multi0=sonnet5 (anthropic/claude-sonnet-5) and Multi1=jev (typesafe/jev-router) as requested
passchecklistno-silent-outagerun.log shows continuous tick progress with 0 LLM failures in 38 turns each no gaps

Decision timeline

What each model saw as its situation and what it ordered, every 8 game-seconds. Click a turn to jump the video there.

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →