evals
Create your eval

02 — Run · Round 1 · Detroit 1972: crime · sonnet5-detroit-s2

Claude Sonnet 5 plays Detroit 1972

Detroit's crime is out of control in 1972. Bring average crime under 60 in 10 years, without emptying the city. Ended Jan 1982: lost, city score 341.

GOAL MISSED seed 2
Claude Sonnet 5
anthropic/claude-sonnet-5

LOST: Average crime 28 (goal < 60), population 3,280 (floor 25,000); engine verdict: won

Population (start → end)76,160 → 3,280 Peak population76,180City score (start → end)285 → 341 Funds at the end$0Crime / pollution / traffic28 / 59 / 0 Zones R / C / I129 / 64 / 88 Power plants (coal / nuclear)2 / 3Police / fire stations15 / 7 Turns (model / failed)40 (40 / 0) Actions applied / rejected98 / 36 Mean turn latency9.6sModel cost$0.718Wall clock6.6 min Jetty run IDc55e7a2b

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 197276,160$20,0002859264
1Apr 197276,180$18,4745399264
2Jul 197272,400$18,46353910367
3Oct 197265,240$16,43853910064
4Jan 197360,720$13,60644110064
5Apr 197355,000$13,5964419364
6Jul 197348,460$12,5724418161
7Oct 197345,060$12,0394418161
8Jan 197440,260$8,7903726861
9Apr 197437,100$8,7903726358
10Jul 197432,820$8,2403726353
11Oct 197430,140$7,7313724953
12Jan 197526,860$4,1833354553
13Apr 197524,540$4,1833354550
14Jul 197522,380$4,1833354350
15Oct 197520,260$4,1833354053
16Jan 197618,380$2,2703224051
17Apr 197616,280$2,2703223855
18Jul 197614,080$2,2703223655
19Oct 197612,520$2,2703223654
20Jan 197711,420$9183153451
21Apr 19779,980$9183153451
22Jul 19779,060$9183153449
23Oct 19778,060$9183153348
24Jan 19787,720$8452933248
25Apr 19786,640$8452933248
26Jul 19785,820$8452933150
27Oct 19785,660$8452933045
28Jan 19795,460$6802933045
29Apr 19795,120$6802933046
30Jul 19794,960$6802933048
31Oct 19794,480$6802933048
32Jan 19803,980$4482772949
33Apr 19803,800$4482772949
34Jul 19803,940$4482772953
35Oct 19804,940$4482772853
36Jan 19814,580$2344042850
37Apr 19813,760$2344042853
38Jul 19813,580$2344042853
39Oct 19813,300$2344042851
40Jan 19823,120$03412859

Final map

Final city map: Claude Sonnet 5 on Detroit 1972: crime

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1982 after 40 turns; scenario verdict True
passcode_checkdecisions-logged40 rows for 40 turns
passcode_checkllm-turns-valid0/40 model turns failed
passcode_checkvideo-decodesduration 36.9s
passcode_checklimits-respected6.6 wall-minutes (model budget 45.0 min)
failjudgescenario-goalLOST: Average crime 28 (goal < 60), population 3,280 (floor 25,000); engine verdict: won (score 0 / threshold 1)
passchecklistnotes-explain-outcomeNotes section explains economic collapse vs crime reduction, cites model thoughts from turns 1, 8, and 40
passchecklistplayer-matches-requestresult.json confirms player=sonnet5 model=anthropic/claude-sonnet-5, task=detroit, seed=2
passchecklistno-silent-outagerun.log shows 0 LLM failures across all 40 turns, no silent stretch

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →