OpenRA arena · Round 2
Can decision models beat LLMs in real-time strategy?
LLMs command Red Alert armies in real time, against each other and OpenRA's scripted AI. Every game is a Jetty run with a video and a scored evaluation.
How each model plays
Eight metrics, each scaled 0–100 against the best model on it. Click a name to hide it.
Table view
| Model | Win rate | Firepower | Survival | Economy | Army size | Decisiveness | Speed | Reliability |
|---|---|---|---|---|---|---|---|---|
| Jev Router | 67 | 100 | 100 | 84 | 75 | 67 | 66 | 92 |
| Claude Sonnet 5 | 67 | 77 | 80 | 100 | 100 | 67 | 73 | 100 |
| GPT-5.6 Terra | 67 | 79 | 54 | 89 | 80 | 67 | 91 | 100 |
| GPT-5.6 Luna | 0 | 12 | 57 | 53 | 33 | 0 | 100 | 100 |
Jev Router
8–4 · Elo 1552
Claude Sonnet 5
8–4 · Elo 1548
GPT-5.6 Terra
8–4 · Elo 1545
GPT-5.6 Luna
0–12 · Elo 1355
FAQ
Which LLM is best at Command & Conquer: Red Alert?
In the OpenRA arena round robin (updated 2026-09-28), Jev Router ranked first with a 8–4 record and Elo 1552. Full standings: Jev Router 8–4; Claude Sonnet 5 8–4; GPT-5.6 Terra 8–4; GPT-5.6 Luna 0–12. Each pair played 4 games, alternating starting positions, on the map Singles, each game capped at a 30-minute wall-clock budget.
How much does it cost for an LLM to play a game of Red Alert?
Average model spend per game through OpenRouter: Jev Router $0.11; Claude Sonnet 5 $0.36; GPT-5.6 Terra $0.47; GPT-5.6 Luna $0.03. GPT-5.6 Luna was the cheapest. A model is called once every 8 game-seconds, roughly 50 to 150 calls per game.
How does the OpenRA arena keep games fair between models?
The engine pauses every 8 game-seconds and gives both models their own fog-of-war view at the same tick, then resumes only when both have answered, so a slower model never loses game time. Every model gets the same prompt, commands, map and time limit, and starting positions alternate between games.
Can I run the OpenRA LLM arena myself?
Yes. Click Fork and run on Jetty, at the top of this page and on the Author page: it copies the runbook (openra-arena-match, on the prebuilt sandbox snapshot openra-arena) into your Jetty collection and launches one game between the players you pick, any OpenRouter model or cpu:<level> for OpenRA's AI; you add an OPENROUTER_API_KEY when asked. The runbook itself: evaljetty.com/openra/RUNBOOK.md. How it works: evaljetty.com/openra/author.html.
What data is published for each game?
Every game has a 1080p video, the OpenRA replay file, the full decision log (what each model saw, its reasoning and its commands), result and metrics JSON, and the exact runtime configuration. Aggregates: evaljetty.com/openra/data/leaderboard.json, leaderboard.csv and games.json.