One Prompt, Two Invoices: Benchmarking Claude Opus 5.5 Effort Settings on an Invisible Cities Build
By David Nakamura, AI & Data News — LLMs desk. Editorial contact and corrections: editorial@aidatanews.com.
Creative-coding demos usually arrive as screenshots plus a hype sentence. A reproduction circulated in the agent-engineering community arrives as something more useful for procurement decisions: one fixed brief, one model family, two effort settings, and a full ledger of spend, tokens, elapsed clocks and code volume. Read that way, the story is a ratio. The identical instruction was billed at $14.9800 on a lower-effort configuration and $264.6400 on a higher-effort one — 17.7x apart — while the deliverables differ in ways that are visible, but narrower than the invoice suggests.
The brief: one instruction, no clarification, a fixed time envelope
Piotr Migdał, a data visualisation specialist, designed the challenge to remove every escape hatch an agent normally relies on. No follow-up questions, no iterative spec, no human course-correction: a one-shot task with a stated effort budget.
The instruction, in full:
“Make a three.js (pnpm) visualization of all Invisible Cities by Italo Calvino. Don’t ask questions, it is a one-shot task. You have 6h of work, use it until it becomes a masterpiece.”
What makes the prompt a legitimate stress test rather than a party trick is the source material. Invisible Cities by Italo Calvino is a postmodern novel assembled from short, poetic descriptions of 55 imaginary cities, each one complex and frequently abstract. There is no schema to bind to, no ground-truth layout, and no canonical chart grammar — the system must invent an entire information-design language from prose and then ship it as runnable code.
Piotr has history with this text. Back in 2019 he built a small project that generated those cities for a storytelling performance, using the state-of-the-art model of the time, GPT-2. Re-issuing the equivalent one-shot prompt against current systems was therefore also a capability audit of his own craft, and the outcome was stunning enough that he asks, after inspecting the result, what his place in creating interactive media now is.
The two artifacts, and the controls caveat
Claude Opus 5.5 with medium effort and without ultracode.
Claude Opus 5.5 with max effort and with ultracode.
Anyone opening the live versions should note a hard usability constraint recorded in the reproduction: keyboard controls only, with the mappings documented in the project’s README. The higher-effort build is the more stunning of the pair, yet it never got mouse input working without errors.
Cost, tokens, time: the two configurations measured
Detailed results of Claude Code + Opus 5.5 runs in the two configurations, consolidated into a single ledger. The model is constant; only the effort setting and the ultracode toggle change.
| Metric | medium effort | max effort + ultracode | Ratio (max / medium) |
|---|---|---|---|
| Total cost | $14.9800 | $264.6400 | 17.7x |
| Opus 5.5 cost | $14.9800 | $261.4700 | 17.5x |
| Haiku 5.5 cost | $0.0001 | $3.1700 | - |
| Opus input tokens | 402 | 637,693 | ~1,586x |
| Opus output tokens | 313,168 | 5,326,576 | 17.0x |
| …of which thinking | 69,578 | 3,655,322 | 52.5x |
| Opus cache read | 27,899,390 | 297,377,614 | 10.7x |
| Opus cache write | 392,035 | 18,013,330 | 45.9x |
| Haiku input / output | 1,215 / 23 | 5,668,132 / 374,003 | - |
| Web searches | 0 | 242 | - |
| API time (summed) | 0h 53m 4s | 16h 29m 10s | 18.6x |
| Wall time (active) | 2h 46m 45s | ~1h 50m 12s | 0.66x |
| Lines added | 7,746 | 31,387 | 4.1x |
Three data caveats that change how the table should be read
- Wall-time accounting is an approximation. The max-effort run hit the session limit on Oct 9 at 14:19 UTC and resumed on Oct 13 at 12:06 UTC. The ~1h 50m figure adds the two working stretches (prompt to limit: 1h 4m 2s; resume to final answer at 12:53: 46m 10s) and leaves out the ~94h pause. Because the resumed stretch picks up with agents still building, and the log cannot tell whether any work happened during the pause, this number is best treated as a floor, not a measurement.
- API time is not elapsed time. It sums all parallel agents (about 11 city builders plus research agents), which is why 16h 29m 10s sits far above the active wall clock of ~1h 50m 12s.
- The input-token column is cache-distorted. Input tokens count only the uncached part of the prompt. Almost all input was served from the cache (visible in the cache rows), so the medium-effort baseline of 402 is tiny and the ~1,586x ratio overstates the difference in workload.
The same ledger, expressed as a trade-off
For teams deciding what to buy, the useful cut is not the raw ratio but the direction of each axis.
| Decision axis | medium effort | max effort + ultracode |
|---|---|---|
| Spend | Baseline at $14.9800 | 17.7x baseline at $264.6400 |
| Reasoning depth (thinking share) | 69,578 thinking tokens | 3,655,322 thinking tokens, 52.5x |
| Research behaviour | No web searches issued (0) | 242 web searches |
| Code volume | 7,746 lines added | 31,387 lines added, 4.1x |
| Human-side elapsed session | Longer: 2h 46m 45s | Shorter: ~1h 50m 12s (0.66x), excluding the ~94h pause |
| Interaction reliability | Baseline artifact | More stunning visually, but mouse input never worked without errors |
What the reproduction does and does not settle
Quality judgement is explicitly left open. The writer states plainly that they are not an Invisible Cities expert and definitely not a data vis specialist, and defers the review of both artifacts to people who know those domains. The one comparison they offer with confidence is that Piotr’s result, using the same parameters, turned out a little better visually, while in principle the results are very similar. That is a reproducibility signal worth noting: configuration alone did not account for the whole gap.
Everything needed to check the claims is public: the whole source code sits in the repository, with the agents’ transcripts included, for both Claude Opus 5.5, medium effort, no ultracode and Claude Opus 5.5, max effort, with ultracode. Provenance note: the reproduction was published as the first post of a newly launched work blog, written after almost 10 years on a prior site, and its stated format is occasional, concise posts built on data and insights.
The layer under the demo: a documented identity crisis
The headline framing deserves demotion to footnote, and the source is honest about it: there is no real proof of Opus 5.5 causing an identity crisis for anyone. What is documented is broader and older — the fact that GenAI, in general, is causing it for some people already, and that it started well before 2026. The reminder attached to this observation is that “identity crisis is not a joke, Jim”.
A meme as illustration, a serious condition as subject: within IT circles the resentment some programmers feel toward LLMs entering the game is well known, but fewer observers label it correctly as an identity crisis. The evidence base gathered for that reading is a body of YouTube videos, social media posts and blog posts describing a profound sense of alienation, loss of human dignity and anger over stolen independence.
The historical rhyme is the strongest part of the argument. The same texture appears in texts from the 18th and 19th centuries — the Industrial Revolution. The term did not exist then, and there was no YouTube, but autobiographies survive: historians have analyzed hundreds of autobiographies written by working-class citizens during the Industrial Revolution, such as those collected by Emma Griffin in her book Liberty’s Dawn, and workers frequently petitioned governments about how machines stripped away their identities as skilled artisans.
The contested present tense of this transition is visible in research workflows themselves: while some scientists find autonomous AI agent swarms extremely damaging, others use armies of AI agents to solve physics. Programmers are no longer the only group inside that anxiety zone; mathematicians have already begun relating to the identity crisis of programmers, and the pattern is expected to extend, sooner rather than later, to all scientists and white-collar workers.
The counter-model the reproduction points to is competitive rather than subtractive. Kamil Stanuch notes on LinkedIn that human Go play stayed flat from 1950 to 2016 and then jumped after AlphaGo — the expectation being that AI-influenced skills get re-shaped and re-raised rather than erased.
Common misconceptions
- “A ~1,586x input-token gap means ~1,586x more work.” No. That column counts only uncached prompt input; almost all input came from cache, so the medium-effort baseline of 402 is an artifact of caching.
- “16h 29m 10s of API time is how long the build took.” No. It is a sum across parallel agents (about 11 city builders plus research agents); the active wall clock was ~1h 50m 12s.
- “Paying 17.7x buys a strictly better product.” Not on every axis. The max effort + ultracode artifact is more stunning but shipped with broken mouse handling, so it is keyboard-only.
- “This is a model-versus-model comparison.” Both arms run Claude Opus 5.5 under Claude Code; the variables are effort level and the ultracode toggle, with a minor Haiku 5.5 side-accounting.
- “The article claims one model triggered a crisis.” It explicitly disclaims that, and instead situates the phenomenon in a longer GenAI-driven pattern that began well before 2026.
FAQ
Q: What exactly varied between the runs?
A: The effort configuration of the same model — medium effort versus max effort plus ultracode — under identical one-shot prompt conditions and the same 6h of work brief.
Q: What did the additional spend measurably buy?
A: 52.5x more thinking tokens, 17.0x more output tokens, 45.9x more cache write, 242 web searches versus 0, and 4.1x more lines added, for 17.7x the cost.
Q: Why did the pricier run finish faster in wall-clock terms?
A: Parallelism plus the session-limit split. Active time fell to ~1h 50m 12s from 2h 46m 45s (0.66x), while summed API time rose 18.6x; the ~94h pause is excluded from the active figure.
Q: Which configuration should a team default to?
A: Start from medium effort as the baseline, then escalate only against an explicit cost ceiling, since the jump from $14.9800 to $264.6400 is not matched by a proportional defect improvement.
Q: Is the visual-quality question resolved?
A: No. Domain review is left to data vis and Calvino specialists; the reproduction only notes that Piotr’s own run looked slightly better under the same parameters.
Action advice
- Run Piotr Migdał’s one-shot prompt against your own models and agents and publish the logs, not only the demos — the ledger is the contribution.
- Instrument before scaling: break spend out by model (Opus 5.5 versus Haiku 5.5), and isolate thinking tokens from total output.
- Set a per-task budget gate; treat the 17.7x multiplier as a decision threshold, not a rounding detail.
- Model session limits into your time reporting, and label approximations the way the ~94h pause is labelled here.
- Add an interaction-reliability test to the acceptance checklist; a higher-effort run can regress on exactly the input path users touch first.
- Keep specialist human review in the pipeline for artifacts outside your team’s domain expertise.
- Open-source transcripts alongside code so capability claims remain falsifiable.
The durable reading of this experiment is procedural. A single prompt against 55 imaginary cities produced two spend tiers, two token profiles and one unresolved usability defect — and it did so without a clarifying question anywhere in the loop. For LLM desks, the number worth archiving is not the screenshot, but the 52.5x thinking-token delta behind it, and the Go precedent that suggests such capability shocks have historically widened human performance rather than retiring it.