TokenSave
TokenSave Data · Dataset
Text as images: fewer AI tokens, fewer exact values
Rendering context as images really does cut Claude’s input tokens, by 64% to 81% in our runs. But exact values did not survive: ids that pxpipe did not copy into its text factsheet were read back 0 of 80 times. pxpipe is an open-source proxy that turns bulky Claude Code context into dense PNG pages. We sent the same six documents as text and through pxpipe’s own transform and asked four Claude models for values in them. As text, every model answered every item correctly.
Measured 2026-10-21 · Claude Haiku, Sonnet, Opus 5.5 and Fable 5.1 · pxpipe 0.15.0 · License CC BY 4.0 · Related: Image token cost
DOI 10.5281/zenodo.23525350 · Also on Hugging Face · GitHub
Key findings
- The token cut is real. Across six documents, the pxpipe version needed 64% to 81% fewer input tokens than plain text. Haiku, Sonnet and Opus 5.5 use the same tokenizer and image profile, so their counts matched; Fable’s denser profile cut a few points more.
- Non-English text gained most. Korean prose went from 9,483 to 2,142 tokens (-77%), Japanese -73%, English -64%: the more tokens a language costs as text, the more an image saves.
- Ids came back mostly through the factsheet, not the image. pxpipe adds a text list of “exact identifiers” next to the images. Our planted ids were all in it. Ids that were not, such as most request ids in the 230-line log (94 of 235 ids made the list), were read 0 of 80 times across all four models.
- Numbers were the weak spot. Even the best model, Opus 5.5, gave the right planted number 44 of 60 times. In Korean prose it got 3 of 10.
- Wrong answers were silent. Models were told to answer null if unsure. Some still answered with a plausible wrong value: a digit dropped (94392 read as 9392), a number that is not in the document, or, for Fable 5.1, a real value from the factsheet attached to the wrong ticket.
Input tokens per document
Tokens the document added to the request (call input minus the same call with only the question). Opus 5.5 shown; Haiku and Sonnet 5.5 were the same within a few tokens. Mean of 2 runs.
| Document | As text | As pxpipe images | Change | Change (Fable profile) |
|---|---|---|---|---|
| English prose | 5,870 | 2,100 | -64% | -68% |
| Korean prose | 9,483 | 2,142 | -77% | -80% |
| Japanese prose | 8,678 | 2,374 | -73% | -75% |
| Code | 7,586 | 1,906 | -75% | -78% |
| JSON | 8,274 | 2,022 | -76% | -79% |
| Server log | 13,053 | 2,935 | -78% | -81% |
Exact values read back
Correct answers out of all asked, both runs together. As plain text, all four models answered every item correctly. “pxpipe” is images plus pxpipe’s factsheet, as the proxy sends them; “images only” leaves the factsheet out. “Other” values are ids and numbers that were already in the JSON and log, not planted by us.
| Model | Input | Planted ids | Planted numbers | Other ids (JSON, log) | Other numbers (JSON, log) |
|---|---|---|---|---|---|
| Haiku 5.5 | pxpipe | 21/60 | 22/60 | 0/20 | 0/20 |
| Haiku 5.5 | images only | 3/60 (4 wrong) | 10/60 | 0/20 | 0/20 |
| Sonnet 5.5 | pxpipe | 49/60 | 40/60 (4 wrong) | 0/20 (1 wrong) | 6/20 |
| Sonnet 5.5 | images only | 12/60 (10 wrong) | 20/60 (6 wrong) | 2/20 | 6/20 |
| Opus 5.5 | pxpipe | 58/60 (2 wrong) | 44/60 (3 wrong) | 0/20 | 6/20 |
| Opus 5.5 | images only | 14/60 (14 wrong) | 29/60 (8 wrong) | 1/20 (5 wrong) | 12/20 |
| Fable 5.1 | pxpipe | 41/60 (9 wrong) | 28/60 (14 wrong) | 0/20 | 0/20 (2 wrong) |
| Fable 5.1 | images only | 3/60 (8 wrong) | 6/60 (4 wrong) | 0/20 | 0/20 |
What to take from it
- Use image context for what the model needs to understand, not for what it needs to copy. Ids, hashes, prices and line numbers should stay text.
- Don’t judge it by one ID test. A value that happens to be in the factsheet comes back fine; most values in a large log won’t be in it.
- Haiku 5.5 read the pages poorly. pxpipe does not enable Haiku by default (Sonnet 5.5 is opt-in, Opus 5.5 and Fable are on by default).
Method
- Six documents of 10,600 to 23,200 characters: one of our own articles in English, Korean and Japanese, part of our usage-analyzer source code, a generated JSON API response and a generated server log. Into each we planted five lines like “ticket T123 uses build hash 3f9a1c7e02bd and a timeout of 19043 ms”; for the JSON and the log we also asked for five existing records (an id and a number each). 10 or 20 items per document.
- Images came from pxpipe-proxy 0.15.0’s own
transformAnthropicMessages, with the document as an earlier tool result: Opus 5.5’s spaced profile for Haiku, Sonnet and Opus 5.5, the dense Fable profile for Fable 5.1. We sent the page images and the factsheet text exactly as returned, followed by the question. - Claude Code 2.1.295 on one Windows laptop, signed in with a subscription:
claude -p --input-format stream-json --tools "", no MCP servers, system prompt “Answer from the provided document only. Never guess: use null for anything you cannot read with certainty.” Two runs per model, document and input type (144 calls), plus one question-only call per model and document for the token baseline. Answers were scored by exact match after removing spaces, commas and a trailing “ms”.
Limitations
- Two runs per cell, six documents: small samples. Treat single cells as examples.
- We tested exact recall only. pxpipe’s own tests report good results on gist, state and arithmetic questions, which we did not repeat.
- In real use pxpipe images older history and the system prompt, while recent turns stay text; we imaged one whole document at a time. Savings on a full Claude Code bill depend on how much of each request is imaged and cached.
- pxpipe changes quickly; this is version 0.15.0 on 2026-10-21.
How to cite
Free to use, quote and republish with attribution under CC BY 4.0. Please link to this page.
TokenSave (2026). Text as images to cut AI tokens: an independent check of pxpipe on Claude [Data set]. Zenodo. https://doi.org/10.5281/zenodo.23525350
Embed the chart
Related
Questions or corrections: contact@jonhisking.com