evals
Create your eval

02 — Run · Round 1 · Grow a city · gpt56terra-grow-s2

GPT-5.6 Terra plays Grow a city

Empty generated map, $20,000, 20 years: reach 10,000 people without going bankrupt. Ended Jan 1920: 3,660 people, city score 322.

GOAL MISSED seed 2
GPT-5.6 Terra
openai/gpt-5.6-terra

population 3,660 (goal 10,000), city score 322

Population (start → end)0 → 3,660 Peak population20,900City score (start → end)500 → 322 Funds at the end$0Crime / pollution / traffic38 / 42 / 2 Zones R / C / I69 / 27 / 17 Power plants (coal / nuclear)3 / 0Police / fire stations1 / 0 Turns (model / failed)80 (80 / 0) Actions applied / rejected222 / 6 Mean turn latency9.6sModel cost$1.540Wall clock13.0 min Jetty run ID476d94f4

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 19000$20,00050000
1Apr 19000$14,10372700
2Jul 1900340$12,8717273560
3Oct 1900960$10,914727947
4Jan 19012,400$3,681441947
5Apr 19013,420$2,6294411936
6Jul 19014,440$2,6294414136
7Oct 19015,420$2,6294414136
8Jan 19024,920$3,3572615036
9Apr 19024,120$3572615744
10Jul 19025,800$482615741
11Oct 19027,760$482615541
12Jan 190310,400$3694026647
13Apr 190313,420$3694026652
14Jul 190316,400$684027852
15Oct 190319,260$6840211566
16Jan 190419,460$1,58020511559
17Apr 190417,240$1,58020514259
18Jul 190415,020$1,58020513359
19Oct 190414,000$1,58020513364
20Jan 190512,760$2,58815011857
21Apr 190511,320$1,05815010357
22Jul 190511,900$1,05815010354
23Oct 190512,420$1,0581508749
24Jan 190614,040$1,4883729349
25Apr 190614,760$1,0833729353
26Jul 190615,160$1,0833729860
27Oct 190614,240$1,08337210158
28Jan 190712,560$2,14728110158
29Apr 190711,160$2,14728110150
30Jul 190710,280$2,1472818248
31Oct 19079,120$2,1472818248
32Jan 19088,360$3,0432287445
33Apr 19087,600$382286548
34Jul 19086,640$382286552
35Oct 19085,580$382286252
36Jan 19095,100$2632525048
37Apr 19094,700$2632525045
38Jul 19094,260$2632524645
39Oct 19094,400$2632524653
40Jan 19104,120$8512704652
41Apr 19103,880$8512704052
42Jul 19103,260$8512703752
43Oct 19102,420$8512703752
44Jan 19111,940$7031373546
45Apr 19111,600$7031373146
46Jul 19112,280$6851373147
47Oct 19113,720$6751373653
48Jan 19122,520$1,0671273753
49Apr 19122,500$1,0671273752
50Jul 19122,240$1,0671273647
51Oct 19122,880$1,0671273447
52Jan 19135,300$1,3583043447
53Apr 19137,340$1,3583044533
54Jul 19138,740$4783046340
55Oct 19139,980$4783046340
56Jan 191413,680$1,2593347643
57Apr 191415,980$1,1393348749
58Jul 191417,800$1,1393348749
59Oct 191418,260$1,03233411253
60Jan 191519,540$83334710066
61Apr 191520,900$83334710064
62Jul 191519,580$83334711164
63Oct 191518,600$83334711259
64Jan 191616,580$2,26125511252
65Apr 191615,180$2,11425510252
66Jul 191614,020$2,1142558951
67Oct 191613,040$2,1142558947
68Jan 191711,960$2,5552608043
69Apr 191710,320$2,5552607043
70Jul 19179,820$2,5552607039
71Oct 19179,000$2,5552606447
72Jan 19188,220$3,5632395347
73Apr 19187,620$5492395343
74Jul 19187,240$5492395142
75Oct 19186,260$5492395442
76Jan 19195,680$1973095443
77Apr 19195,000$1973094636
78Jul 19194,400$1973094238
79Oct 19194,020$1973094238
80Jan 19203,660$03223842

Final map

Final city map: GPT-5.6 Terra on Grow a city

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1920 after 80 turns; scenario verdict None
passcode_checkdecisions-logged80 rows for 80 turns
passcode_checkllm-turns-valid0/80 model turns failed
passcode_checkvideo-decodesduration 35.6s
passcode_checklimits-respected13.0 wall-minutes (model budget 45.0 min)
failjudgecity-goalpopulation 3,660 (goal 10,000), city score 322 (score 3,660 / threshold 10,000)
passchecklistnotes-explain-outcomeNotes section cites model thoughts from turns 1, 62 (peak), and 80 and explains the energy-supply / funding collapse that caused the decline
passchecklistplayer-matches-requestresult.json confirms player=gpt56terra model=openai/gpt-5.6-terra task=grow seed=2
passchecklistno-silent-outagerun.log shows 80 consecutive turns 0/80 failures no gaps

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →