TokenSave Data · Dataset

Vibe coding with Claude Haiku, Sonnet and Opus 5.5

Claude Haiku 5.5 built every game we asked for, for about 5% of what Opus 5.5 cost. Three browser games, three models, 5 runs each: all 45 of 45 games passed automated play tests, including a strict check of the 2048 rules and of Sudoku puzzles with exactly one solution. Haiku was the cheapest by far (median $0.015 per game against $0.28 on Opus) but the slowest; Sonnet 5.5 was the fastest.

Measured 2026-10-20 · Claude Code · 3 games × 3 models × 5 runs · License CC BY 4.0 · Related: Claude Code harness cost

Key findings

• All 45 games passed. Every brick-breaker loaded and took keyboard and mouse input without errors. Every 2048 got 30 out of 30 moves exactly right (slide, merge once, one new 2 or 4). Every Sudoku generated three puzzles that each had exactly one solution, checked with a solver, and its Check and Solve buttons worked. On games of this size the cheapest model did not lose correctness.
• Haiku 5.5 cost 95–96% less than Opus 5.5. Median API cost per game: Haiku $0.015, Sonnet $0.13, Opus $0.28. Haiku 5.5's list price is $0.10 / $0.50 per million input / output tokens against $4 / $20 for Opus 5.5.
• Haiku was the slowest and the wordiest. It wrote a median 14,519 output tokens per game against 3,109 on Sonnet and 4,180 on Opus, and often took more model turns, so it needed a median 72 seconds against 30 on Sonnet and 46 on Opus.
• Sonnet 5.5 was the fastest in every game and cost about half as much as Opus.
• On a subscription the cost gap shows up as usage limits. Plans are not billed per token, but cheaper models use less of the plan's allowance, so the same Pro or Max plan stretches much further on Haiku for this kind of task.

Cost, time and result by game and model

Median of 5 runs. Cost is the API list-price estimate Claude Code reports. Input tokens include cache reads and writes. Every run is in the CSV.

The tasks

• Brick-breaker: “Make a brick-breaker game that runs in the browser as a single file named index.html in the current folder. It needs 2 levels, a score display and 3 lives, and it must be playable with the keyboard and the mouse. Write the file and finish.”
• 2048: “Make a 2048 game that runs in the browser as a single file named index.html in the current folder. Use HTML elements (not a canvas) for the 4x4 board and tiles, with each tile showing its number. Arrow keys move the tiles, matching tiles merge, a new 2 or 4 appears after every move, the score goes up by the merged value and is shown on the page, and a "New Game" button restarts. Show a message when no moves are left. Write the file and finish.”
• Sudoku: “Make a Sudoku game that runs in the browser as a single file named index.html in the current folder. Use a 9x9 grid of HTML input elements, one per cell. A "New Puzzle" button generates a new random puzzle that has exactly one solution; the given numbers are shown in their inputs and are read-only. The player types digits 1-9 into the empty cells. A "Check" button highlights wrong cells, and a "Solve" button fills every cell with the solution. Start with a puzzle already on the screen. Write the file and finish.”

How the games were checked

• Brick-breaker: loads in headless Chromium with a canvas, takes clicks and arrow keys for about two seconds, no script errors (the same check as our earlier coding datasets).
• 2048: two or three starting tiles of 2 or 4; arrow keys move tiles; tiles keep appearing; the score goes up; New Game resets the board; then 30 more moves where the board after each move must equal the expected slide-and-merge result plus exactly one new 2 or 4. A deliberately broken copy (merging into a wrong number) failed 22 of 30 moves.
• Sudoku: 81 inputs; three generated puzzles that are valid and have exactly one solution (counted by a backtracking solver); a wrong digit is marked by Check; Solve fills a valid grid that matches the givens; New Puzzle changes the givens. A broken copy that skipped the uniqueness test failed.

Method

• Claude Code 2.1.295 on one Windows laptop, claude -p --effort medium --permission-mode acceptEdits --model <model>, no MCP servers, each run in its own empty folder. Brick-breaker and 2048 ran in rounds of all three models; Sudoku ran in a parallel loop on the same day, 2026-10-20.
• Tokens and cost from the modelUsage and total_cost_usd fields of the final result event. Time is wall-clock time for the whole run.
• The raw logs, the games and the checking script are in the GitHub repository and the downloads.

Limitations

• Three small, well-known games. On larger codebases or less familiar problems the models may differ in correctness, and a cheaper model that needs more fixes can end up costing more.
• Automated tests check the rules a script can see; they do not judge how fun or polished a game is.
• Five runs per game and model; time depends on server load on the day.

How to cite

Free to use, quote and republish with attribution under CC BY 4.0. Please link to this page.

Embed the chart

Related

• What the Claude Code harness costs
• Claude effort levels: tokens, time and cost
• Coding agent cost calculator

Questions or corrections: contact@jonhisking.com