evals
Create your eval

02 — Run · Round 1 · Detroit 1972: crime · jev-detroit-s2

Jev Router plays Detroit 1972

Detroit's crime is out of control in 1972. Bring average crime under 60 in 10 years, without emptying the city. Ended Jan 1982: lost, city score 456.

The model used its 45-minute budget by turn 31; the city ran on untouched from there to the deadline.

GOAL MISSED seed 2
Jev Router
typesafe/jev-router

LOST: Average crime 40 (goal < 60), population 10,040 (floor 25,000); engine verdict: won

Population (start → end)76,160 → 10,040 Peak population76,160City score (start → end)285 → 456 Funds at the end$0Crime / pollution / traffic40 / 48 / 0 Zones R / C / I129 / 64 / 90 Power plants (coal / nuclear)3 / 1Police / fire stations12 / 7 Turns (model / failed)40 (30 / 4) Actions applied / rejected122 / 15 Mean turn latency91.5sModel cost$3.107Wall clock45.9 min Jetty run ID85fe296a

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 197276,160$20,0002859268
1Apr 197275,120$15,9255379268
2Jul 197271,260$13,3755376366
3Oct 197265,800$13,3705376763
4Jan 197360,860$9,9814726763
5Apr 197354,380$8,9634726263
6Jul 197349,840$8,4374725260
7Oct 197345,300$8,4374725260
8Jan 197442,020$6,9093924653
9Apr 197439,780$6,8813924254
10Jul 197436,360$6,8813924252
11Oct 197433,680$6,8813924052
12Jan 197529,900$5,2804103654
13Apr 197528,800$5,2804103651
14Jul 197527,620$5,2404106651
15Oct 197525,560$4,7324105552
16Jan 197623,180$4,4504025550
17Apr 197621,220$4,4284025350
18Jul 197619,600$1,4174025150
19Oct 197617,800$1,4174025147
20Jan 197716,360$2,0173134846
21Apr 197714,520$2,0173134546
22Jul 197712,800$1,9423134546
23Oct 197711,480$1,9423134443
24Jan 197810,840$1,2453164243
25Apr 19789,740$1,0603164245
26Jul 19789,120$1,0363163945
27Oct 19788,360$1,0363163844
28Jan 19797,820$7963773844
29Apr 197910,040$7963773846
30Jul 197914,020$4133773850
31Oct 197917,040$4133773850
32Jan 198019,140$1736884251
33Apr 198017,960$1736884755
34Jul 198015,840$1736884749
35Oct 198014,280$1736884649
36Jan 198112,820$05164451
37Apr 198111,540$05164450
38Jul 198110,780$05164150
39Oct 198110,380$05164048
40Jan 198210,040$04564048

Final map

Final city map: Jev Router on Detroit 1972: crime

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1982 after 40 turns; scenario verdict True
passcode_checkdecisions-logged40 rows for 40 turns
passcode_checkllm-turns-valid4/30 model turns failed
passcode_checkvideo-decodesduration 36.9s
passcode_checklimits-respected45.9 wall-minutes (model budget 45.0 min)
failjudgescenario-goalLOST: Average crime 40 (goal < 60), population 10,040 (floor 25,000); engine verdict: won (score 0 / threshold 1)
passchecklistnotes-explain-outcomesummary.md Notes section explains city arc citing model thoughts from turns 1, 15, and 30
passchecklistplayer-matches-requestresult.json shows player=jev, task=detroit, seed=2, matching requested parameters
passchecklistno-silent-outageretry run shows 4 scattered FAILED turns (4, 8, 23, 28) with no consecutive stretches; llm-turns-valid passed at 13.3% failure rate

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →