Low-Resource Languages in NLP: What "Low-Resource" Means and What Works

Multilingual & low-resource · Published 2026-10-26

A wooden printer's type case full of metal type pieces for the Cherokee syllabary, scattered in its compartments — Cherokee syllabary type at the New Echota historic site, Georgia, where the Cherokee Phoenix newspaper was printed from 1828. US National Park Service, public domain.

Machine translation, speech recognition and chat assistants work best in a few dozen languages. For the great majority of the world's languages, they work poorly or not at all. "Low-resource" is the term NLP uses for this situation; it describes the data available to researchers, not the richness of the languages or the size of their communities.

What makes a language "low-resource"?

The term is relative and slippery. A language with tens of millions of speakers can be low-resource for NLP if little of its text is digitised, while a small European language with a well-funded standardisation body may have excellent resources. Researchers usually look at several kinds of resources:

Unlabelled text for pretraining: books, news, web pages, social media.
Parallel text, translations aligned sentence by sentence with another language, essential for machine translation (see our guide to parallel corpora).
Annotated data: texts labelled for parts of speech, named entities, sentiment and so on.
Lexical resources such as dictionaries and wordnets.
Speech data with transcriptions, crucial for languages that are mainly spoken.
Tools and standards: keyboards, fonts, spell-checkers, a stable orthography.

Pratik Joshi and colleagues proposed a widely cited classification in "The State and Fate of Linguistic Diversity and Inclusion in the NLP World" (ACL 2020). They sorted languages into six classes, from 0 (virtually no labelled or unlabelled data) to 5 (abundant resources and strong representation in research), and found that the large majority of the world's languages fall into the lowest class, while research attention is concentrated on the few at the top.

Why the web does not solve the problem

Large multilingual models are trained on text crawled from the web, and some crawls claim to cover hundreds of languages. The quality of that coverage is another matter. Julia Kreutzer and a large team of co-authors, many of them speakers of the languages concerned, manually inspected samples from five public multilingual datasets in "Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets" (TACL 2022). For many smaller languages, a large share of the sampled sentences were not usable text in the labelled language at all: they were in another language, machine-generated, boilerplate or fragments. Some corpora contained essentially no usable text.

Language identification is part of the problem: automatic identifiers confuse closely related languages and short texts, and are themselves trained on limited data. Tokenizers learned mostly from large languages also break smaller languages into many fragments, making them more expensive and harder for models to learn, as our article on tokenization explains.

Methods that help

Cross-lingual transfer

A model pretrained on many languages at once can transfer some of what it learns from high-resource languages to lower-resource ones, particularly related languages that share vocabulary or structure. XLM-R (Conneau and colleagues, ACL 2020), trained on filtered web text in 100 languages, showed strong transfer, but also documented the "curse of multilinguality": with a fixed model size, adding more languages eventually lowers performance for each.

Making the most of monolingual text

Back-translation, introduced for neural MT by Rico Sennrich and colleagues in 2016, turns monolingual text in the target language into synthetic parallel data by translating it automatically into the source language. It is now a standard technique when parallel data is scarce.

Small, high-quality data

Experience from low-resource projects suggests that small, carefully translated or annotated datasets can matter more than much larger noisy crawled ones. Good evaluation data matters most of all: without a reliable test set, no one can tell whether a system works. The FLORES evaluation benchmarks, created by Meta with professional translators, extended to 200 languages for the "No Language Left Behind" translation project (2022, published in Nature in 2024).

Communities, not just data

A recurring lesson is that building language technology for a community works best with that community. Masakhane, a grassroots network of African researchers founded in 2019, described its approach in "Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages" (Findings of EMNLP 2020), in which participants built translation benchmarks for dozens of African languages, many involving people who were not professional researchers. Steven Bird, in "Decolonising Speech and Language Technology" (COLING 2020), argued that technologies should be designed around what language communities want, which may be transcription tools, learning materials or archives rather than general-purpose translators.

This raises ethical questions about who owns and controls language data, especially for Indigenous languages. The United Nations proclaimed 2022–2032 the International Decade of Indigenous Languages, coordinated by UNESCO, which includes digital empowerment among its themes.

In Europe

Low-resource is not only a concern far from Europe. Regional and minority languages such as Basque, Breton, Sámi languages or Romani have far fewer digital resources than the EU's large official languages, and even some official languages lag well behind English. European programmes and infrastructures that address this are described in our article on Europe's language technology. The everyday consequences show up whenever people depend on translation tools in unfamiliar languages, for example when reading a rental contract after moving to another country.

If you work on a low-resource language

Start with evaluation: build a small, clean test set with native speakers before training anything.
Audit any crawled data by reading samples; check language identification and remove machine-translated or duplicate text.
Use related languages for transfer, and pick a multilingual model whose tokenizer handles your script reasonably.
Involve speakers as partners and authors, agree on data ownership and licences, and credit contributors.
Publish data and documentation so others can build on your work rather than start again.

Sources

Joshi, Santy, Budhiraja, Bali and Choudhury, "The State and Fate of Linguistic Diversity and Inclusion in the NLP World", ACL 2020.
Kreutzer and colleagues, "Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets", TACL 2022.
Conneau and colleagues, "Unsupervised Cross-lingual Representation Learning at Scale", ACL 2020.
Sennrich, Haddow and Birch, "Improving Neural Machine Translation Models with Monolingual Data", ACL 2016.
NLLB Team, "Scaling neural machine translation to 200 languages", Nature, 2024.
Nekoto and colleagues (Masakhane), "Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages", Findings of EMNLP 2020.
Bird, "Decolonising Speech and Language Technology", COLING 2020.
Eberhard, Simons and Fennig (eds.), Ethnologue: Languages of the World, SIL International (annual editions).