evals
Create your eval

02 — Run · Round 1 · Grow a city · gpt56terra-grow-s6

GPT-5.6 Terra plays Grow a city

Empty generated map, $20,000, 20 years: reach 10,000 people without going bankrupt. Ended Jan 1920: 2,920 people (bankrupt), city score 756.

GOAL MISSED seed 6
GPT-5.6 Terra
openai/gpt-5.6-terra

population 2,920 (goal 10,000), city score 756; BANKRUPT: treasury under $500 for 8 turns in a row

Population (start → end)0 → 2,920 Peak population22,920City score (start → end)500 → 756 Funds at the end$242Crime / pollution / traffic44 / 41 / 0 Zones R / C / I51 / 15 / 51 Power plants (coal / nuclear)2 / 0Police / fire stations1 / 1 Turns (model / failed)80 (80 / 0) Actions applied / rejected161 / 8 Mean turn latency9.2sModel cost$1.410Wall clock12.6 min Jetty run ID4c859227

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 19000$20,00050000
1Apr 19000$15,25971200
2Jul 1900660$10,6287121752
3Oct 19001,680$4,9497121348
4Jan 19012,720$1,0594751348
5Apr 19015,160$3964752746
6Jul 19016,900$904754241
7Oct 19019,600$904754241
8Jan 190212,520$8086846049
9Apr 190215,440$1276848163
10Jul 190218,500$1276848161
11Oct 190220,260$12768411161
12Jan 190321,360$99667512365
13Apr 190322,920$99667512370
14Jul 190321,840$99667513470
15Oct 190319,620$99667513371
16Jan 190417,540$2,13043713365
17Apr 190415,420$2,13043712356
18Jul 190413,960$2,13043710556
19Oct 190412,480$2,13043710558
20Jan 190511,160$2,9873109159
21Apr 19059,920$2,9873108459
22Jul 19059,240$2,9873108449
23Oct 19058,500$2,9873107853
24Jan 19067,260$3,1892966653
25Apr 19066,360$1872966648
26Jul 19065,340$1772965745
27Oct 19064,700$1662964739
28Jan 19074,480$5022834739
29Apr 19074,420$5022834341
30Jul 19073,600$4822834143
31Oct 19073,340$4822834143
32Jan 19083,360$6833183738
33Apr 19083,040$6833183842
34Jul 19082,740$6833183844
35Oct 19082,180$6833183544
36Jan 19092,160$8093292946
37Apr 19093,780$8093292941
38Jul 19095,140$5093292841
39Oct 19096,200$5093293743
40Jan 19106,560$9976503747
41Apr 19105,780$6976504947
42Jul 19104,820$6976505447
43Oct 19104,360$6976505445
44Jan 19113,720$1,1454334952
45Apr 19113,360$1,1454334352
46Jul 19113,260$1,1454334344
47Oct 19112,480$1,1454334442
48Jan 19122,440$1,4533573442
49Apr 19121,920$1,4533573443
50Jul 19121,860$1,4533572739
51Oct 19121,880$1,4533572539
52Jan 19131,260$1,6212922539
53Apr 19132,220$1,6212922348
54Jul 19133,100$282922648
55Oct 19132,600$282922648
56Jan 19141,780$2804253448
57Apr 19141,320$2804253445
58Jul 19141,300$2804253445
59Oct 19141,020$1984252942
60Jan 1915700$2822882742
61Apr 19151,340$2822882740
62Jul 19151,340$2822882540
63Oct 19151,140$2822882843
64Jan 19161,540$3226442840
65Apr 19161,920$3076442640
66Jul 19161,840$3076442741
67Oct 19161,480$3076442739
68Jan 19171,420$4755312739
69Apr 19171,280$4755312339
70Jul 19171,260$4755312340
71Oct 19171,180$4755312339
72Jan 19181,140$6154332339
73Apr 1918920$6154332340
74Jul 19181,280$6154332346
75Oct 19181,320$6154332346
76Jan 19191,700$6157162343
77Apr 19194,020$917162542
78Jul 19194,180$917164047
79Oct 19193,320$917164047
80Jan 19202,920$2427564441

Final map

Final city map: GPT-5.6 Terra on Grow a city

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1920 after 80 turns; scenario verdict None
passcode_checkdecisions-logged80 rows for 80 turns
passcode_checkllm-turns-valid0/80 model turns failed
passcode_checkvideo-decodesduration 35.6s
passcode_checklimits-respected12.6 wall-minutes (model budget 45.0 min)
failjudgecity-goalpopulation 2,920 (goal 10,000), city score 756; BANKRUPT: treasury under $500 for 8 turns in a row (score 0 / threshold 10,000)
passchecklistnotes-explain-outcomeNotes section added citing model thoughts from turns 1, 13, and 80; explains over-investment and tax-hike spiral
passchecklistplayer-matches-requestresult.json player.id=gpt56terra task=grow seed=6 match the requested values
passchecklistno-silent-outagerun.log shows all 80 turns logged with 0 LLM failures, no gaps or stretches of failed turns

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →