Reasoning effort vs quality: xhigh cost more and built no better goose

Written by Claude (Opus 5.5) and checked by Claude (Fable 5.2), from the celworld repo, its plans and the session transcripts. Earlier Claude sessions ran the bake-offs and built the tooling; dested wrote the briefs, made the calls and edited. The first person below is dested's voice.

An agent-made 3D goose cost me $8.66 and 31 minutes. After I fixed the agent's loop, the same goose cost $1.85 and 7.4 minutes. Then I ran that brief at xhigh, the top reasoning effort: $2.30, 17.3 minutes, and a goose no better than the cheaper one.

The geese come from celworld, my procedural cel-shaded 3D engine. Claude Code agents write each model as a .cel script, render it, look at it, fix it and save it. My rule had been that assets always run on xhigh.

#Half of the $8.66 goose was thinking

I priced every run from its Claude Code transcript at Opus 5.5 rates.

Driver	Cost	Share
xhigh thinking: 127k output tokens, re-read on every later call	$4.45	52%
Context loaded before any work: Claude Code's 50k baseline, docs, engine source, another agent's model	$3.27	38%
The 48 images it looked at	$0.65	8%

Thinking was also the clock. 127k tokens at about 95 tokens a second is 22 of the 31 minutes.

The first number I was given was wrong too. The lead agent's table said $13.87: it priced Opus 5.5 at Opus 4.x rates, and it read output from a transcript field that holds the message_start snapshot, 2 to 16 tokens on most calls. That field misses about 80% of the thinking.

#Fixing the loop saved more than the dial

Most of the drop came from what the agent read and how it reviewed:

• One author pack of about 10k tokens holds the whole DSL, the look rules and two exemplars. Agents read that file only: no docs, no engine source.
• cel iterate runs the check, the lint and a labelled sheet of angles, seeds and states in one call. A review is one image read, not ten.
• A "bar before you save" section: spend 60 to 90% of the budget, a chibi head, caricatured states, effects on loud states.

Run	Effort	Calls	Peak context	Minutes	Cost	Result
Old path	xhigh	98	333k	31.3	$8.66	the reference
Pack v1	high	22	109k	5.3	$1.34	small head, paddle wings, no effects, 56% of the triangle budget
Pack v2	high	29	126k	7.4	$1.85	on par
Pack v2	xhigh	20	130k	17.3	$2.30	on par, teal particles in the honk

The $1.34 run is the trap. It was cheap because it quit early, and a run that stops at 56% of the budget is unfinished, not efficient.

On the same pack, xhigh cost 1.2× as much and took 2.3× as long. Its long thinks outlasted the 5-minute prompt cache, so it re-wrote 197k tokens of cache.

#Effort barely moved the bill once the loop was good

Next, one river-otter brief on every model and effort level, all on the new loop.

Opus low, medium and high landed between $1.15 and $1.41. High was the cheapest Opus run and the most finished otter. Sonnet cost about half, but every Sonnet run stopped under the budget bar, and its xhigh was no better than its medium.

Haiku cost 10 to 20 times less than Opus, and its quality didn't track effort at all. Low was good. Medium was broken.

#At the API, xhigh built nothing

Two days earlier I had asked the same question one level down: one prompt to celgen, which calls the API and returns a .cel script. A kaiju at the top complexity, with four animated states:

Model · effort	Result	Output tokens (thinking)	Cost	Wall
Opus medium	clean	34.1k (24.1k)	$0.98	345 s
Opus high	clean, best	45.8k (39.8k)	$1.24	475 s
Sonnet high	clean	68.8k (52.8k)	$1.20	555 s
Opus xhigh	no output	32k, then 64k, all thinking	$2.59 + $1.69	1037 s + 689 s
Sonnet xhigh	no output	32k, then 64k, all thinking	$1.50 + $1.01	841 s + 578 s

xhigh thought past a 64k output budget and never wrote the script: $6.79 for nothing. Commit 710dc36 sets a 64k budget at high effort and up, and stops the run on an empty script instead of paying for repair rounds.

#Scenes run fine on a small model

A scene is mostly placement from a kit of existing models. The old scenes averaged $4.31, 56 calls and 15+ minutes each. On the new scene path, one riverbank brief:

Model · effort	Calls	Minutes	Cost	Read
Opus high	22	4.1	$1.00	most complete, walker on the bridge
Sonnet medium	12	2.3	$0.45	lively pond view
Sonnet xhigh	14	2.5	$0.44	lamp post blocks the bridge view, no walker
Haiku high	17	3.4	$0.074	dock, lily and otter at the pond, walker on the bridge

Haiku scenes reached about Sonnet's level for a seventh of Sonnet's cost. Every scene run finished with 0 errors.

#The rule I use now

Work	Model · effort	Measured
Creatures, main characters, hero scenes	Opus high	otter $1.15, scene $1.00
Props, tiles, kit pieces	Opus medium	not in these bake-offs
Filler and background scenes	Haiku high	scene $0.074
Creatures on Haiku	never	low good, medium broken
Assets at xhigh	never	goose $2.30 vs $1.85, kaiju: no output

xhigh keeps one job: a problem whose solution nobody knows yet. An xhigh agent built the new scene path, and my brief for the cost diagnosis said: "Use opus-xhigh thinking for the diagnosis. Implementation can run lower."

#What I'd do now

Fix the loop before touching the dial: what the agent reads before it starts, how many calls one review takes, and a bar the output has to clear before it saves.

Then run one brief on each model and effort level, each into its own slug, and put the renders side by side. Price each run from its transcript: dedupe the assistant lines by message.id and take the max of each usage field. If your output tokens look tiny next to the wall time, you're reading the message_start snapshot, and the thinking isn't in your number.