evals
Run evals on Jetty

02 — Run · Round 2 · Round robin · r2-rr-sonnet5-gpt56luna-g3

Claude Sonnet 5 vs GPT-5.6 Luna

Claude Sonnet 5 destroyed the enemy base after 7.2 game-minutes · 10.1 min wall clock

WIN ukraine · west
Claude Sonnet 5
anthropic/claude-sonnet-5
Value destroyed$13,700Value lost$1,600 Units killed / lost19 / 16Buildings killed / lost7 / 0 Peak army$6,700Orders issued210 Decision turns (failed)55 (1)Mean latency4.5sModel cost$0.390
LOSS russia · east
GPT-5.6 Luna
openai/gpt-5.6-luna
Value destroyed$1,600Value lost$13,700 Units killed / lost16 / 19Buildings killed / lost0 / 7 Peak army$2,500Orders issued59 Decision turns (failed)55 (0)Mean latency3.6sModel cost$0.044

Evaluation

passchecks 10/10 verdict computed from the gating checks

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, content, key)
passsteplaunchLaunch run-match as a detached process
passstepsuperviseSupervise in the foreground until run-match exits
passsteprenderRender the 1080p recording
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checkgame-has-winnersonnet5 won: base destroyed
passcode_checkdecisions-loggedturns per slot: {'Multi0': 55, 'Multi1': 55}
passcode_checkllm-turns-validfailed/total per slot: {'Multi0': '0/55', 'Multi1': '0/55'}
passcode_checkvideo-decodesduration 74.4s
passcode_checkvideo-completevideo ends at tick 10860 of 10860
passcode_checklimits-respectedended after 10.1 wall-minutes (budget 30.0 min): base destroyed
passchecklistnotes-explain-winNotes section explains Claude Sonnet 5 won by destroying base with superior army; cites both models final thoughts from decisions.jsonl
passchecklistplayers-match-requestresult.json confirms Multi0=sonnet5 (anthropic/claude-sonnet-5) and Multi1=gpt56luna (openai/gpt-5.6-luna) matching the requested players
passchecklistno-silent-outageall 55 decision ticks logged at regular 200-tick intervals with zero LLM failures for both players

Decision timeline

What each model saw as its situation and what it ordered, every 8 game-seconds. Click a turn to jump the video there.

Ready to stop guessing if your outputs are good?

Get Started Book a 15-minute walkthrough
Connect your agent to Jetty →