evals
Create your eval

02 — Run · Round 2 · test · r2-smoke-gpt6luna

GPT-6 Luna vs Jev Router

Jev Router destroyed the enemy base after 10.1 game-minutes · 16.1 min wall clock

LOSS russia · west
GPT-6 Luna
openai/gpt-6-luna
Value destroyed$2,100Value lost$22,000 Units killed / lost19 / 37Buildings killed / lost0 / 11 Peak army$2,000Orders issued86 Decision turns (failed)76 (0)Mean latency3.1sModel cost$0.027
WIN ukraine · east
Jev Router
typesafe/jev-router
Value destroyed$22,000Value lost$2,100 Units killed / lost37 / 19Buildings killed / lost11 / 0 Peak army$10,700Orders issued99 Decision turns (failed)76 (13)Mean latency5.1sModel cost$0.162

Evaluation

passchecks 10/10 verdict computed from the gating checks

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, content, key)
passsteplaunchLaunch run-match as a detached process
passstepsuperviseSupervise in the foreground until run-match exits
passsteprenderRender the 1080p recording
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checkgame-has-winnerjev won: base destroyed
passcode_checkdecisions-loggedturns per slot: {'Multi0': 76, 'Multi1': 76}
passcode_checkllm-turns-validfailed/total per slot: {'Multi0': '0/76', 'Multi1': '8/76'}
passcode_checkvideo-decodesduration 102.5s
passcode_checkvideo-completevideo ends at tick 15078 of 15078
passcode_checklimits-respectedended after 16.1 wall-minutes (budget 30.0 min): base destroyed
passchecklistnotes-explain-winsummary.md Notes section cites both models final thoughts explaining how Jev Router won by destroying all enemy buildings
passchecklistplayers-match-requestresult.json records Multi0=gpt6luna (openai/gpt-6-luna) and Multi1=jev (typesafe/jev-router) matching the requested players
passchecklistno-silent-outagerun.log shows continuous decision ticks every 200 ticks with no long gaps; Jev Router had 8 failures but they are within the 20% threshold and no silent stretches

Decision timeline

What each model saw as its situation and what it ordered, every 8 game-seconds. Click a turn to jump the video there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →