evals
Create your eval

02 — Run · Round 1 · Detroit 1972: crime · gpt56luna-detroit-s2

GPT-5.6 Luna plays Detroit 1972

Detroit's crime is out of control in 1972. Bring average crime under 60 in 10 years, without emptying the city. Ended Jan 1982: lost, city score 704.

GOAL MISSED seed 2
GPT-5.6 Luna
openai/gpt-5.6-luna

LOST: Average crime 30 (goal < 60), population 15,280 (floor 25,000); engine verdict: won

Population (start → end)76,160 → 15,280 Peak population76,320City score (start → end)285 → 704 Funds at the end$6Crime / pollution / traffic30 / 49 / 2 Zones R / C / I146 / 64 / 95 Power plants (coal / nuclear)2 / 3Police / fire stations18 / 7 Turns (model / failed)40 (40 / 0) Actions applied / rejected241 / 60 Mean turn latency7.9sModel cost$0.087Wall clock5.4 min Jetty run IDaba6000b

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 197276,160$20,0002859268
1Apr 197276,320$16,3245379268
2Jul 197272,940$15,0355379566
3Oct 197267,580$13,79753710264
4Jan 197361,240$10,28445810264
5Apr 197357,280$9,2354588663
6Jul 197351,960$9,0354587959
7Oct 197347,140$7,4244587959
8Jan 197442,460$3,7173866954
9Apr 197439,220$3,0533866351
10Jul 197435,420$2,9333866351
11Oct 197432,260$2,8233865851
12Jan 197528,820$1,6833455347
13Apr 197525,540$1,6833455343
14Jul 197523,000$1,1373454743
15Oct 197520,620$1,1023454346
16Jan 197618,200$2653194345
17Apr 197616,160$2503194043
18Jul 197614,460$1863193843
19Oct 197612,480$1813193840
20Jan 197710,840$03073745
21Apr 19779,740$03073545
22Jul 19777,880$03073544
23Oct 19776,560$03073342
24Jan 19786,600$03093342
25Apr 19786,280$03093344
26Jul 19786,700$03093244
27Oct 19789,760$03093445
28Jan 19799,640$3236543445
29Apr 19798,580$3236543648
30Jul 19797,700$1146543547
31Oct 19796,620$106543547
32Jan 19805,680$1864643448
33Apr 19805,220$1194643251
34Jul 19804,460$134643253
35Oct 19804,400$134643153
36Jan 19814,260$1604093053
37Apr 19813,760$954093052
38Jul 19814,260$224093052
39Oct 19819,340$64093048
40Jan 198216,440$67043049

Final map

Final city map: GPT-5.6 Luna on Detroit 1972: crime

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1982 after 40 turns; scenario verdict True
passcode_checkdecisions-logged40 rows for 40 turns
passcode_checkllm-turns-valid0/40 model turns failed
passcode_checkvideo-decodesduration 36.9s
passcode_checklimits-respected5.4 wall-minutes (model budget 45.0 min)
failjudgescenario-goalLOST: Average crime 30 (goal < 60), population 15,280 (floor 25,000); engine verdict: won (score 0 / threshold 1)
passchecklistnotes-explain-outcomeNotes section added to summary.md citing model thoughts from turns 1, 8, and 40
passchecklistplayer-matches-requestresult.json confirms player=gpt56luna, model=openai/gpt-5.6-luna, task=detroit, seed=2
passchecklistno-silent-outage0 LLM failures across all 40 turns, run.log shows only the expected scenario-goal JSON at exit

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →