Language shapes thought, and not just for humans.Why does o1 sometimes think in Chinese?
After the release of o1, some users noticed an intriguing phenomenon: during reasoning, the model occasionally switches to… Chinese
(example). The final answer is always provided in the user’s language, but the reasoning process remains a mystery.
Interestingly, o1 isn’t the only reasoning model with this quirk. QwQ also switches to Chinese mid-reasoning. However, QwQ’s behavior is more understandable—it originates from China, and its training data is predominantly in Chinese. But why would o1, developed by OpenAI, do the same?
OpenAI has not commented on this behavior, but experts and researchers have theories. For example, the CEO of Hugging Face suggests (and it’s the most straightforward explanation) that the model was trained on a massive amount of Chinese data. Many companies, possibly including OpenAI, use Chinese annotation services, which could explain this bias.
But this raises another question: why only Chinese? The training data surely included vast amounts of text in Hindi, Thai, or Spanish. Yet, the model never switches to these languages. Why?
Some speculate this might be an intentional OpenAI experiment. Chinese tokens carry more information per token compared to other languages, making reasoning in Chinese potentially shorter and cheaper. If we consider that the model performs a kind of solution space search, reasoning in certain languages may yield correct answers more efficiently (possibly due to data imbalance), leading the model to favor these “branches.”
For now, this behavior remains a mystery. Hopefully, OpenAI will eventually shed some light on it.
#AI