evals
Run evals on Jetty

02 — Run · Round 2 · Round robin · r2-rr-jev-sonnet5-g4

Claude Sonnet 5 vs Jev Router

Claude Sonnet 5 destroyed the enemy base after 5.6 game-minutes · 8.5 min wall clock

WIN ukraine · west
Claude Sonnet 5
anthropic/claude-sonnet-5
Value destroyed$9,000Value lost$1,200 Units killed / lost10 / 6Buildings killed / lost5 / 0 Peak army$4,800Orders issued88 Decision turns (failed)42 (0)Mean latency4.2sModel cost$0.255
LOSS england · east
Jev Router
typesafe/jev-router
Value destroyed$1,200Value lost$9,000 Units killed / lost6 / 10Buildings killed / lost0 / 5 Peak army$2,400Orders issued58 Decision turns (failed)42 (0)Mean latency5.0sModel cost$0.099

Evaluation

passchecks 10/10 verdict computed from the gating checks

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, content, key)
passsteplaunchLaunch run-match as a detached process
passstepsuperviseSupervise in the foreground until run-match exits
passsteprenderRender the 1080p recording
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checkgame-has-winnersonnet5 won: base destroyed
passcode_checkdecisions-loggedturns per slot: {'Multi0': 42, 'Multi1': 42}
passcode_checkllm-turns-validfailed/total per slot: {'Multi0': '0/42', 'Multi1': '0/42'}
passcode_checkvideo-decodesduration 57.8s
passcode_checkvideo-completevideo ends at tick 8362 of 8362
passcode_checklimits-respectedended after 8.5 wall-minutes (budget 30.0 min): base destroyed
passchecklistnotes-explain-winNotes section cites final thoughts from both sonnet5 and jev and explains the infantry push that destroyed Jevs base
passchecklistplayers-match-requestresult.json confirms Multi0=sonnet5 (Claude Sonnet 5) west and Multi1=jev (Jev Router) east as requested
passchecklistno-silent-outagerun.log shows uninterrupted 200-tick decision turns from tick 1 to 8201 with zero LLM failures in both slots

Decision timeline

What each model saw as its situation and what it ordered, every 8 game-seconds. Click a turn to jump the video there.

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →