AI models cost more when you don’t use English
Rest of World/iStock
People who use less widely spoken languages could end up paying more to use AI than those who speak English or other dominant languages.
A new study published last month by the Coalition for Language Equity in AI Research found AI models require far more tokens for some languages than others. Tokens are the units of text or data that the models process. The researchers have called this a hidden “tokenizer tax,” which can make AI more expensive in languages with smaller online presence. A tokenizer is a tool that breaks down raw text, images, or audio into tokens.
Even within the 20 most popular languages on the internet, there is a gap in token usage, the research found. The most expensive of the languages used between 1.5 and 5 times more tokens than the cheapest. This gap widened when researchers expanded the set to 200 languages, with the token cost gap in one AI model growing by 17 times.
Dispatches from the ground 🌏︎
This week, Chinese AI lab Moonshot curbs the hacking capabilities of its latest model, and the UAE inks a series of AI partnerships with France.
China
Chinese AI company Moonshot deliberately limited the hacking capabilities of its latest AI model as the industry grows more concerned about the risks of AI misuse. In a blog post announcing Kimi K3.1 Pro, its latest open-weight model, Moonshot said it had removed certain cyber-related data from the training process to make the model less capable of discovering and exploiting cyber vulnerabilities. AI models are now able to expose cyber vulnerabilities much faster than humans, leading to fears of a surge in cyberattacks. The Chinese government has also recently proposed export bans on AI models with high cyber capabilities. Moonshot’s decision reflects a growing awareness among Chinese AI labs that they need to prevent their open models, which tend to lack the safety filters of proprietary models, from being misused by dangerous actors.
— Viola Zhou from New York City
Middle East
The United Arab Emirates and France signed a series of agreements tied to artificial intelligence on October 20 during French President Emmanuel Macron’s visit to Abu Dhabi. French AI company AMI Labs will work with Abu Dhabi’s Mohamed bin Zayed University of Artificial Intelligence; G42, the emirate’s biggest AI and technology company; and computing company Analog on joint research, cloud computing, and developing AI systems that can control machines and carry out physical tasks. Abu Dhabi investment firm MGX is also working with French partners to build computing facilities in France, which would allow French companies and researchers to run AI systems locally rather than depending on U.S. cloud providers. Until recently, the UAE was spending billions to bring foreign AI technology and computing power into the Gulf. It is now doing the opposite, using its sovereign wealth to buy and build the facilities that other countries need to run AI. In July, MGX spent $40 billion to acquire data centers across the United States, Brazil, Mexico, and Chile.
— Indranil Ghosh from Abu Dhabi
Stat of the Day
— The number of teachers the UAE plans to train in AI for lesson planning, teaching, and assessment, as it mandates AI education across all grade levels starting in kindergarten. While Western school districts in cities like New York debate restrictions on generative tools in elementary classrooms, Gulf nations are treating AI literacy like reading and basic math, betting early exposure will give students a competitive edge.
And one more thing...
Finnish officials halted the construction of Google’s data center after the company illegally felled 250,000 trees without permits, driving displaced wolves into nearby villages and drawing criticism from U.S. President Donald Trump over offshoring tech capital, The Guardian reported. The incident captures the growing friction between Silicon Valley expansionism and local populations, as author Paris Marx wrote in Rest of World: Communities worldwide are pushing back against hyperscale data centers that promise economic prosperity but end up consuming critical resources and damaging local ecosystems.
Thanks for reading! Send us your feedback at [email protected]. If you enjoyed this newsletter, please forward it to a friend — Edited by Tolu Olasoji
Powered by beehiiv