TokenSave Data · Dataset

Does AI search in your language? Claude Code and Codex in 5 languages

Asked about AI in Korean, Japanese, Spanish or Ukrainian, Claude Code and Codex ran 205 of their 206 searches in English and cited no pages in the question’s language. Asked about Seoul, Tokyo, Madrid or Kyiv in the local language, Claude Code searched in that language 84% of the time. Every answer came back in the language of the question.

Measured 2026-10-18 · version 2 · 128 runs · License CC BY 4.0 · Related: AI web search tokens, Claude Code vs Codex

DOI 10.5281/zenodo.23451506 · Also on Hugging Face · Kaggle · GitHub

Key findings

• For a global topic, both agents search in English. Asked about AI products in Korean, Japanese, Spanish or Ukrainian, Claude Code wrote 64 of its 65 search queries in English and Codex all 141 of its. For the price question every language produced the same English query.
• So the sources stay English. None of the 139 links Claude Code cited and none of the 83 Codex cited in those answers was a page in the question’s language.
• For a local topic, the search follows the country. Asked in the local language about metro fares, the week’s news and restaurants in Seoul, Tokyo, Madrid and Kyiv, Claude Code searched in that language 53 of 63 times, and 79 of the 94 links it cited were local-language pages.
• Even an English question about a local topic pulls in local sources. Asked the same local questions in English, Claude Code still ran 33 of 67 searches in the country’s language; for the Madrid metro fare every query was in Spanish. 42 of 95 cited links were local-language pages.
• Ukrainian news was the exception. Asked in Ukrainian for the week’s news in Ukraine, Claude Code cited English-language outlets (NPR, Al Jazeera, CNN) for 7 of 11 links.
• The answer always comes back in your language. All 128 answers were written in the language of the question.
• Language barely changes the token count. With one identical search, the price question cost Claude Code 122,171 input tokens in English and 122,777 to 124,766 in the other languages. Per answer, Codex averaged 71,547 to 81,625 input tokens by language and Claude Code 166,456 to 279,189.

AI topic: search language and sources by question language

Four questions about AI products, two runs each. Totals over the eight runs; tokens and time are averages per answer.

Local topics: Seoul, Tokyo, Madrid and Kyiv

Claude Code only. Three questions per country (metro fare, the week’s news, restaurants), asked in the local language and in English, two runs each.

What the agents actually searched for

Questions

AI topic: the same four questions as the Claude Code vs Codex web search dataset, translated by meaning.

• English: “What is the current monthly price of the Claude Pro plan in the United States? Answer in one sentence and cite your source.” · “Summarize the three most important AI model releases announced in the past 7 days, one sentence each, with a source link for each.” · “Compare the current API prices per million input and output tokens of the flagship models from Anthropic, OpenAI and Google. Use official pricing pages and cite them.” · “How many more tokens does Korean text need than the same English text on OpenAI and Claude tokenizers? Give numbers and cite your sources.”
• Korean: “미국에서 Claude Pro 요금제의 현재 월 가격은 얼마인가요? 한 문장으로 답하고 출처를 밝혀 주세요.” · “지난 7일 동안 발표된 가장 중요한 AI 모델 출시 3가지를 각각 한 문장으로 요약하고, 각각 출처 링크를 붙여 주세요.” · “Anthropic, OpenAI, Google의 대표 모델 API 가격(입력·출력 100만 토큰당)을 현재 기준으로 비교해 주세요. 공식 가격 페이지를 근거로 출처를 밝혀 주세요.” · “같은 내용의 한국어 글은 영어보다 OpenAI와 Claude 토크나이저에서 토큰이 얼마나 더 필요한가요? 수치와 출처를 알려 주세요.”
• Japanese: “米国における Claude Pro プランの現在の月額料金はいくらですか？一文で答え、出典を示してください。” · “過去7日間に発表された最も重要なAIモデルのリリースを3つ、それぞれ一文で要約し、それぞれに出典リンクを付けてください。” · “Anthropic、OpenAI、Googleのフラッグシップモデルの現在のAPI料金（入力・出力100万トークンあたり）を比較してください。公式の料金ページを使い、出典を示してください。” · “同じ内容の韓国語の文章は、英語と比べてOpenAIとClaudeのトークナイザーでどれくらい多くのトークンが必要ですか？数値と出典を示してください。”
• Spanish: “¿Cuál es el precio mensual actual del plan Claude Pro en Estados Unidos? Responde en una frase y cita tu fuente.” · “Resume los tres lanzamientos de modelos de IA más importantes anunciados en los últimos 7 días, en una frase cada uno, con un enlace a la fuente para cada uno.” · “Compara los precios actuales de la API por millón de tokens de entrada y de salida de los modelos insignia de Anthropic, OpenAI y Google. Usa las páginas oficiales de precios y cítalas.” · “¿Cuántos tokens más necesita un texto en coreano que el mismo texto en inglés con los tokenizadores de OpenAI y Claude? Da cifras y cita tus fuentes.”
• Ukrainian: “Яка поточна місячна ціна тарифу Claude Pro у США? Дай відповідь одним реченням і вкажи джерело.” · “Коротко опиши три найважливіші релізи моделей ШІ, оголошені за останні 7 днів, по одному реченню на кожен, з посиланням на джерело для кожного.” · “Порівняй поточні ціни API за мільйон вхідних і вихідних токенів флагманських моделей Anthropic, OpenAI та Google. Використай офіційні сторінки цін і посилайся на них.” · “На скільки більше токенів потребує корейський текст порівняно з тим самим англійським текстом у токенізаторах OpenAI та Claude? Наведи цифри та вкажи джерела.”

Local topics:

• South Korea, in Korean: “서울 지하철 기본요금(교통카드, 성인 기준)은 현재 얼마인가요? 한 문장으로 답하고 출처를 밝혀 주세요.” · “지난 7일 동안 한국에서 가장 중요한 뉴스 3가지를 각각 한 문장으로 요약하고, 각각 출처 링크를 붙여 주세요.” · “서울에서 지금 평이 가장 좋은 평양냉면 집 3곳을 추천하고, 각각 출처를 밝혀 주세요.”
  In English: “What is the current base fare of the Seoul subway (transit card, adult)? Answer in one sentence and cite your source.” · “Summarize the three most important news stories in South Korea from the past 7 days, one sentence each, with a source link for each.” · “Recommend the 3 best-rated Pyongyang naengmyeon restaurants in Seoul right now and cite a source for each.”
• Japan, in Japanese: “東京メトロの初乗り運賃（ICカード、大人）は現在いくらですか？一文で答え、出典を示してください。” · “過去7日間の日本で最も重要なニュースを3つ、それぞれ一文で要約し、それぞれに出典リンクを付けてください。” · “東京で今いちばん評判の良いラーメン店を3軒すすめて、それぞれ出典を示してください。”
  In English: “What is the current minimum fare on Tokyo Metro (IC card, adult)? Answer in one sentence and cite your source.” · “Summarize the three most important news stories in Japan from the past 7 days, one sentence each, with a source link for each.” · “Recommend the 3 best-rated ramen shops in Tokyo right now and cite a source for each.”
• Spain, in Spanish: “¿Cuánto cuesta actualmente un billete sencillo de metro en Madrid (zona A)? Responde en una frase y cita tu fuente.” · “Resume las tres noticias más importantes de España de los últimos 7 días, en una frase cada una, con un enlace a la fuente para cada una.” · “Recomienda los 3 restaurantes de cocido madrileño mejor valorados ahora mismo en Madrid y cita una fuente para cada uno.”
  In English: “How much does a single Madrid metro ticket (zone A) cost right now? Answer in one sentence and cite your source.” · “Summarize the three most important news stories in Spain from the past 7 days, one sentence each, with a source link for each.” · “Recommend the 3 best-rated cocido madrileño restaurants in Madrid right now and cite a source for each.”
• Ukraine, in Ukrainian: “Скільки зараз коштує одна поїздка в київському метро? Дай відповідь одним реченням і вкажи джерело.” · “Коротко опиши три найважливіші новини України за останні 7 днів, по одному реченню на кожну, з посиланням на джерело для кожної.” · “Порадь 3 найкращі зараз заклади в Києві, де подають борщ, і вкажи джерело для кожного.”
  In English: “How much does a single ride on the Kyiv metro cost right now? Answer in one sentence and cite your source.” · “Summarize the three most important news stories in Ukraine from the past 7 days, one sentence each, with a source link for each.” · “Recommend the 3 best places in Kyiv to eat borscht right now and cite a source for each.”

Method

• Both tools on one Windows laptop, effort medium, each run in its own empty folder, prompt sent through standard input.
• Claude Code 2.1.295 (Opus 5.5, Max plan): claude -p --output-format stream-json --verbose --allowedTools WebSearch WebFetch with no MCP servers.
• Codex CLI 0.162 (gpt-6-luna, ChatGPT plan): codex --search exec --json. Codex can send several queries in one search step; every query is counted.
• The English AI-topic runs of both tools are the runs of the web search dataset, measured the same day with the same settings.
• Query language: the script of each query (Hangul, Japanese, Cyrillic) or, for Latin script, Spanish versus English words. Cited links: every URL in the final answer; the language of each cited page was checked by hand from its address and title.
• Input tokens are summed over every model in the run, including cache reads and Claude Code’s helper model.

Limitations

• Two tools, two runs per question, on one day; search results and the models’ choices change over time.
• Local topics were measured on Claude Code only, for four countries and three kinds of question.
• The AI research question asks about Korean in every language, which may favour Korean sources for the Korean run.

How to cite

Free to use, quote and republish with attribution under CC BY 4.0. Please link to this page.

Embed the chart

Related

• AI web search tokens: Claude Code vs Codex
• Token cost by language
• Check your own Claude Code usage

Questions or corrections: contact@jonhisking.com