Same prompt, 41 languages: GPT token cost compared

Jonhisking · Oct 1, 2026

I translated one ordinary customer-support prompt into 41 languages and counted tokens with o200k_base, the tokenizer behind GPT-4o and every newer OpenAI model (the last column uses cl100k, the GPT-4-era tokenizer). English needs 34 tokens. The same request costs anywhere from 1.03× (Simplified Chinese) to 2.44× (Punjabi).

The English prompt:

Please summarize the customer email below in three bullet points and suggest a polite reply. The customer says the order arrived two days late and one item was missing from the box.

All 41 languages

Why some languages cost more

Tokenizers are trained mostly on English, so common English words are a single token (" polite", " customer"). Words in other languages are split into pieces, and endings or diacritics often become tokens of their own:

• Czech: navrhněte → nav | r | hn | ě | te
• Polish: opóźnieniem → op | ó | ź | n | ieniem
• Hindi: सारांशित → सार | ांश | ित

The old tokenizer was much worse

On the GPT-4-era tokenizer (cl100k), the gap was far larger. The biggest improvements:

• Malayalam: 10.09× → 1.85×
• Kannada: 9.35× → 1.79×
• Telugu: 9.38× → 2.03×
• Gujarati: 7.18× → 1.59×
• Tamil: 8.47× → 1.97×

That is why the common advice "Korean costs 2–3× English" is out of date: Korean went from 2.50× to 1.44×.

What it costs

With a model at $2 per million input tokens, sending this prompt one million times costs $68 in English, $98 in Korean, $122 in Japanese and $136 in Czech. If the model also answers in that language, the same multiplier applies to output tokens, which usually cost 4–5× more.

How to spend fewer tokens

• Write the system prompt and fixed instructions in English; keep only user input in the user's language.
• Ask for intermediate steps (classification, extraction, tool calls) in English or JSON, and only the final answer in the user's language.
• Cache the fixed part of the prompt (prompt caching).

Limitations

• One prompt only; with other text the ratios move by about ±0.1–0.2.
• Claude and Gemini use different tokenizers, so these numbers apply to OpenAI models.
• Translations started from machine translation and were checked.

Read it in your language

Čeština · Polski · 日本語 · 한국어 · 简体中文 · 繁體中文 · Español · Português · Français · Deutsch · Italiano · Русский · Українська · Türkçe · العربية · فارسی · हिन्दी · Bahasa Indonesia · Tiếng Việt · ไทย · Nederlands · বাংলা · اردو · Filipino · Svenska · עברית · Ελληνικά · Română · Magyar · Dansk · Suomi · Norsk · Slovenčina · मराठी · ગુજરાતી · ಕನ್ನಡ · മലയാളം · தமிழ் · తెలుగు · ਪੰਜਾਬੀ

The original write-up and discussion are on DEV.