evals
Create your eval

02 — Run · Round 1 · Tokyo 1957: monster attack · jev-tokyo-s1

Jev Router plays Tokyo 1957

A monster attacks Tokyo in 1957. Rebuild and get the city score above 500 within 5 years. Ended Jan 1962: lost, city score 372.

GOAL MISSED seed 1
Jev Router
typesafe/jev-router

LOST: City score 372 (goal > 500); engine verdict: lost

Population (start → end)113,100 → 23,120 Peak population113,100City score (start → end)500 → 372 Funds at the end$0Crime / pollution / traffic19 / 44 / 4 Zones R / C / I133 / 91 / 74 Power plants (coal / nuclear)4 / 1Police / fire stations17 / 6 Turns (model / failed)20 (20 / 4) Actions applied / rejected70 / 4 Mean turn latency42.9sModel cost$1.212Wall clock14.4 min Jetty run ID96f24e6b

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 1957113,100$20,0005003160
1Apr 1957107,960$18,9986703160
2Jul 1957104,960$18,9986707465
3Oct 195799,380$18,9986707865
4Jan 195894,840$16,5546337865
5Apr 195889,360$16,5546337467
6Jul 195883,620$11,5156339465
7Oct 195879,000$11,5006339465
8Jan 195973,600$9,2534916563
9Apr 195967,720$9,2534916262
10Jul 195962,940$8,7534916262
11Oct 195956,740$8,2434914662
12Jan 196049,800$4,5724203660
13Apr 196046,140$4,5464203654
14Jul 196041,660$4,5354203154
15Oct 196037,660$1,5294202753
16Jan 196133,620$2133742749
17Apr 196130,500$1823742252
18Jul 196127,000$1573742152
19Oct 196124,560$1573742148
20Jan 196222,760$03721944

Final map

Final city map: Jev Router on Tokyo 1957: monster attack

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1962 after 20 turns; scenario verdict False
passcode_checkdecisions-logged20 rows for 20 turns
passcode_checkllm-turns-valid4/20 model turns failed
passcode_checkvideo-decodesduration 30.0s
passcode_checklimits-respected14.4 wall-minutes (model budget 45.0 min)
failjudgescenario-goalLOST: City score 372 (goal > 500); engine verdict: lost (score 0 / threshold 1)
passchecklistnotes-explain-outcomeNotes section explains the scenario loss with citations from turns 1, 8, and 20 of decisions.jsonl
passchecklistplayer-matches-requestresult.json shows player=jev task=tokyo seed=1 matching the requested parameters
passchecklistno-silent-outagerun.log shows 4 FAILED turns out of 20 (20%), each isolated and within llm-turns-valid threshold; no long stretch of failures

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →