OpenAI’s AI reasoning model ‘thinks’ in Chinese sometimes and no one really knows why – TechCrunch
Latest
AI
Amazon
Apps
Biotech & Health
Climate
Cloud Computing
Commerce
Crypto
Enterprise
EVs
Fintech
Fundraising
Gadgets
Gaming
Government & Policy
Hardware
Layoffs
Media & Entertainment
Meta
Microsoft
Privacy
Robotics
Security
Social
Space
Startups
TikTok
Transportation
Venture
Events
Startup Battlefield
StrictlyVC
Newsletters
Podcasts
Videos
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
Shortly after OpenAI released o1, its first “reasoning” AI model, people began noting a curious phenomenon. The model would sometimes begin “thinking” in Chinese, Persian, or some other language — even when asked a question in English.Given a problem to sort out — e.g. “How many R’s are in the word ‘strawberry?’” — o1 would begin its “thought” process, arriving at an answer by performing a series of reasoning steps. If the question was written in English, o1’s final response would be in English. But the model would perform some steps in another language before drawing its conclusion.“[o1] randomly started thinking in Chinese halfway through,” one user on Reddit said.“Why did [o1] randomly start thinking in Chinese?” a different user asked in a post on X. “No part of the conversation (5+ messages) was in Chinese.”Why did o1 pro randomly start thinking in Chinese? No part of the conversation (5+ messages) was in Chinese… very interesting… training data influence pic.twitter.com/yZWCzoaiit— Rishab Jain (@RishabJainK) January 9, 2025 OpenAI hasn’t provided an explanation for o1’s strange behavior — or even acknowledged it. So what might be going on? Well, AI experts aren’t sure. But they have a few theories. Several on X, including Hugging Face CEO Clément Delangue, alluded to the fact that reasoning models like o1 are trained on datasets containing a lot of Chinese characters. Ted Xiao, a researcher at Google DeepMind, claimed that companies including OpenAI use third-party Chinese data labeling services, and that o1 switching to Chinese is an example of “Chinese linguistic influence on reasoning.”“[Labs like] OpenAI and Anthropic utilize [third-party] data labeling services for PhD-level reasoning data for science, math, and coding,” Xiao wrote in a post on X. “[F]or expert labor availability and cost reasons, many of these data providers are based in China.”Labels, also known as tags or annotations, help models understand and interpret data during the training process. For example, labels to train an image recognition model might take the form of markings around objects or captions referring to each person, place, or object depicted in an image.Studies have shown that biased labels can produce biased models. For example, the average annotator is more likely to label phrases in African-American Vernacular English (AAVE), the informal grammar used by some Black Americans, as toxic, leading AI toxicity detectors trained on the labels to see AAVE as disproportionately toxic.Other experts don’t buy the o1 Chinese data labeling hypothesis, however. They point out that o1 is just as likely to switch to Hindi, Thai, or a language other than Chinese while teasing out a solution.Rather, these experts say, o1 and other reasoning models might simply be using languages they find most efficient to achieve an objective (or hallucinating).“The model doesn’t know what language is, or that languages are different,” Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, told TechCrunch. “It’s all just text to it.”Indeed, models don’t directly process words. They use tokens instead. Tokens can be words, such as “fantastic.” Or they can be syllables, like “fan,” “tas,” and “tic.” Or they can even be individual characters in words — e.g. “f,” “a,” “n,” “t,” “a,” “s,” “t,” “i,” “c.”Like labeling, tokens can introduce biases. For example, many word-to-token translators assume a space in a sentence denotes a new word, despite the fact that not all languages use spaces to separate words.Tiezhen Wang, a software engineer at AI startup Hugging Face, agrees with Guzdial that reasoning models’ language inconsistencies may be explained by associations the models made during training.“By embracing every linguistic nuance, we expand the model’s worldview and allow it to learn from the full spectrum of human knowledge,” Wang wrote in a post on X. “For example, I prefer doing math in Chinese because each digit is just one syllable, which makes calculations crisp and efficient. But when it comes to topics like unconscious bias, I automatically switch to English, mainly because that’s where I first learned and absorbed those ideas.”Wang’s theory is plausible. Models are probabilistic machines, after all. Trained on many examples, they learn patterns to make predictions, such as how “to whom” in an email typically precedes “it may concern.”But Luca Soldaini, a research scientist at the nonprofit Allen Institute for AI, cautioned that we can’t know for certain. “This type of observation on a deployed AI system is impossible to back up due to how opaque these models are,” they told TechCrunch. “It’s one of the many cases for why transparency in how AI systems are built is fundamental.”Short of an answer from OpenAI, we’re left to muse about why o1 thinks of songs in French but synthetic biology in Mandarin.TechCrunch has an AI-focused newsletter! Sign up here to get it in your inbox every Wednesday.Topics
Senior Reporter, Enterprise
Bluesky is getting its own photo-sharing app, Flashes
Meta execs obsessed over beating OpenAI’s GPT-4 internally, court filings reveal
Google’s NotebookLM had to teach its AI podcast hosts not to act annoyed at humans
TikTok users’ attempted migration to Chinese app RedNote isn’t going too well
ChatGPT now lets you schedule reminders and recurring tasks
DOJ confirms FBI operation that mass-deleted Chinese malware from thousands of US computers
OpenAI’s AI reasoning model ‘thinks’ in Chinese sometimes and no one really knows why
Subscribe for the industry’s biggest tech newsEvery weekday and Sunday, you can get the best of TechCrunch’s coverage.TechCrunch’s AI experts cover the latest news in the fast-moving field.Every Monday, gets you up to speed on the latest advances in aerospace.Startups are the core of TechCrunch, so get our best coverage delivered weekly.By submitting your email, you agree to our Terms and Privacy Notice.© 2024 Yahoo.
