evals
Run evals on Jetty
Pilot cup · cup1-sf2-g1

Claude Sonnet 5 vs Gemini 3.8 Flash

Claude Sonnet 5 destroyed the enemy base after 5.2 game-minutes · 6.2 min wall clock

Pilot game (run locally, before the Jetty runs) Replay (.orarep) Decision log Runtime config
WIN ukraine · west
Claude Sonnet 5
anthropic/claude-sonnet-5
Value destroyed$9,400Value lost$1,100 Units killed / lost14 / 1Buildings killed / lost5 / 0 Peak army$4,500Orders issued89 Decision turns (failed)40 (0)Mean latency4.4sModel cost$0.248
LOSS france · east
Gemini 3.8 Flash
google/gemini-3.8-flash
Value destroyed$1,100Value lost$9,400 Units killed / lost1 / 14Buildings killed / lost0 / 5 Peak army$2,200Orders issued37 Decision turns (failed)40 (0)Mean latency2.6sModel cost$0.128

Decision timeline

What each model saw as its situation and what it ordered, every 8 game-seconds. Click a turn to jump the video there.

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.