evals
Create your eval

02 — Run · Round 1 · San Francisco 1906: earthquake · gpt56terra-sanfrancisco-s1

GPT-5.6 Terra plays San Francisco 1906

An earthquake hits San Francisco in 1906. Be a Metropolis (100,000 people) within 5 years. Ended Jan 1911: lost, city score 39.

GOAL MISSED seed 1
GPT-5.6 Terra
openai/gpt-5.6-terra

LOST: Population 47,680 (goal >= 100,000); engine verdict: lost

Population (start → end)107,540 → 47,680 Peak population107,540City score (start → end)633 → 39 Funds at the end$0Crime / pollution / traffic80 / 48 / 14 Zones R / C / I143 / 55 / 59 Power plants (coal / nuclear)6 / 0Police / fire stations10 / 7 Turns (model / failed)20 (20 / 0) Actions applied / rejected186 / 11 Mean turn latency19.2sModel cost$0.646Wall clock6.4 min Jetty run IDd5eb8764

The city over time

Population

People, after every turn

Funds

Treasury, after every turn

City score

0–1000, re-evaluated each January

Table view
TurnDatePopulationFundsCity scoreCrimePollution
0Jan 1906107,540$20,0006337155
1Apr 1906103,220$15,9436627155
2Jul 190697,940$13,23366212257
3Oct 190694,540$11,95866212957
4Jan 190791,040$5,18433112957
5Apr 190789,620$2,29233112257
6Jul 190787,260$71633112057
7Oct 190785,520$70633112057
8Jan 190883,940$1,37916511852
9Apr 190879,940$43416511454
10Jul 190879,640$1616511453
11Oct 190877,880$1616511453
12Jan 190978,140$1,0648211256
13Apr 190976,640$2088211252
14Jul 190976,020$1008210552
15Oct 190972,460$1008210451
16Jan 191065,540$4,3284110451
17Apr 191060,500$1,293419752
18Jul 191053,300$1,020418852
19Oct 191049,680$662418849
20Jan 191147,660$0398048

Final map

Final city map: GPT-5.6 Terra on San Francisco 1906: earthquake

Evaluation

failchecks 9/10 verdict: valid run (code checks and checklist) and task goal met (judge)

StatusKindCheckDetail
passstepenvironmentEnvironment check (snapshot, engine, key)
passsteplaunchLaunch run-city as a detached process
passstepsuperviseSupervise in the foreground until run-city exits
passsteprenderTimelapse video rendered
passstepnotesAppend Notes to summary.md
passcode_checkoutputs-existall present
passcode_checksim-reached-deadlineended Jan 1911 after 20 turns; scenario verdict False
passcode_checkdecisions-logged20 rows for 20 turns
passcode_checkllm-turns-valid0/20 model turns failed
passcode_checkvideo-decodesduration 30.0s
passcode_checklimits-respected6.4 wall-minutes (model budget 45.0 min)
failjudgescenario-goalLOST: Population 47,680 (goal >= 100,000); engine verdict: lost (score 0 / threshold 1)
passchecklistnotes-explain-outcomesummary.md has a Notes section with three cited model thoughts from turns 1, 9, and 20
passchecklistplayer-matches-requestresult.json confirms player=gpt56terra task=sanfrancisco seed=1 as requested
passchecklistno-silent-outagerun.log shows 20/20 turns logged with 0 llm_failures, no gaps or silent failures

Decision timeline

What the mayor thought and did each turn. Click a turn to jump the timelapse there.

Ready to stop guessing if your outputs are good?

Create your eval Book a 15-minute walkthrough
Connect your agent to Jetty →