Multilingual Lexical Knowledge Bases: WordNets, BabelNet and Wikidata Explained

Multilingual & low-resource · Published 2026-10-26

The domed main reading room of the Library of Congress, with concentric rings of desks around a central circulation desk — The Main Reading Room of the Library of Congress, Washington. Photo: Carol M. Highsmith, public domain.

Dictionaries list words; lexical knowledge bases connect meanings. They record that a "car" is a kind of "motor vehicle", that it has "wheels", and that the same concept is "Auto" in German and "macchina" in colloquial Italian. Built over four decades by linguists, volunteers and algorithms, they remain useful companions to neural language models. This guide explains the main ones.

The WordNet model

WordNet began at Princeton University in 1985 under the psychologist George A. Miller, initially as a test of theories about how people store words in memory. Its central unit is the synset, a set of words that can express the same concept in some context, such as {car, auto, automobile, machine, motorcar}. Synsets are connected by semantic relations: hypernymy ("a car is a motor vehicle"), meronymy ("a wheel is part of a car"), antonymy for adjectives, and others. Each synset has a short definition (a gloss) and often example sentences.

The English WordNet contains well over a hundred thousand synsets and has been freely available under a permissive licence for decades, which is one reason it became the standard sense inventory in NLP research, including for word sense disambiguation. Its fine-grained distinctions, however, were designed by lexicographers rather than for applications, and its structure reflects English.

Going multilingual

EuroWordNet and the interlingual index

EuroWordNet, an EU-funded project that ran from 1996 to 1999 under the coordination of Piek Vossen at the University of Amsterdam, built wordnets for several European languages, including Dutch, Italian, Spanish, German, French, Czech and Estonian, and connected them through an Inter-Lingual Index: a list of language-independent concept records to which each language's synsets could be linked. This made it possible to move from a word in one language to its equivalents in others via shared concepts, while letting each wordnet keep relations specific to its language. The Global WordNet Association, founded in 2000, now coordinates wordnet builders around the world.

The Open Multilingual Wordnet

Francis Bond and colleagues gathered openly licensed wordnets for many languages into the Open Multilingual Wordnet, all linked to Princeton WordNet's synsets. In "Linking and Extending an Open Multilingual Wordnet" (ACL 2013), Bond and Ryan Foster extended its coverage to many more languages using data from Wiktionary and the Unicode Common Locale Data Repository. A Collaborative Interlingual Index was later proposed so that concepts missing from English could be added, a long-standing limitation of English-anchored designs.

BabelNet: wordnets meet encyclopedias

Wordnets are good at common words and poor at names, technical terms and new vocabulary, which are exactly what encyclopedias cover. Roberto Navigli and Simone Paolo Ponzetto at Sapienza University of Rome presented "BabelNet: Building a Very Large Multilingual Semantic Network" (ACL 2010), which automatically linked WordNet synsets to Wikipedia pages, used Wikipedia's links between language editions to gather lexicalisations in many languages, and used machine translation to fill some remaining gaps. The resulting "Babel synsets" combine dictionary senses and named entities across languages. Later versions integrated further sources such as Wiktionary and Wikidata and many more languages. BabelNet is distributed under its own BabelNet Non-Commercial License, which limits use to non-commercial research by research institutions; commercial use requires a separate licence, so licence conditions should be checked for each project.

Automatic construction brings breadth but also errors: a wrong link between a sense and a page, or a poor machine translation, propagates into every language connected to it. Users should treat automatically derived entries with appropriate caution, particularly for smaller languages.

Wikidata and ConceptNet

Wikidata, launched by Wikimedia in October 2012, is a collaboratively edited knowledge base of items (people, places, concepts) and statements about them, with labels in hundreds of languages. Its data is released under CC0, which makes it unusually easy to reuse (see our guide to dataset licences). Since 2018 it has also stored lexemes, records of words with their forms and senses, contributed by volunteers in many languages, including small ones.

ConceptNet, which grew out of the Open Mind Common Sense project at the MIT Media Lab around 1999, records everyday relations such as "a knife is used for cutting" in many languages, with data gathered from crowdsourcing and other resources. It is released under a ShareAlike licence.

What they are used for

Word sense disambiguation and entity linking: mapping a word or name in text to a precise entry.
Cross-lingual retrieval: finding documents in other languages through shared concept identifiers rather than translation.
Lexicography and language documentation: building dictionaries for languages that lack them, starting from a shared concept list.
Bootstrapping resources for low-resource languages, by projecting relations from better-resourced languages and asking speakers to correct them.
Grounding language models: giving a model access to structured, checkable facts and definitions, for example by retrieving entries at query time.
Inducing translations: combined with parallel corpora to align senses and terms across languages.

Limits to keep in mind

Anglocentrism. When an English inventory is the hub, concepts lexicalised in other languages but not in English are easily lost.
Uneven coverage. Large languages have rich entries; many languages have only a few thousand, often automatically derived.
Cultural bias. Definitions and hierarchies encode the views of their editors; old taxonomies can contain offensive or outdated categories, which some projects have had to review.
Licences differ. From CC0 to non-commercial terms; combined resources inherit the strictest conditions.

Sources

Miller, "WordNet: A Lexical Database for English", Communications of the ACM 38(11), 1995; Fellbaum (ed.), WordNet: An Electronic Lexical Database, MIT Press, 1998.
Vossen (ed.), EuroWordNet: A Multilingual Database with Lexical Semantic Networks, Kluwer, 1998.
Bond and Foster, "Linking and Extending an Open Multilingual Wordnet", ACL 2013 (ACL Anthology P13-1133).
Navigli and Ponzetto, "BabelNet: Building a Very Large Multilingual Semantic Network", ACL 2010; extended version in Artificial Intelligence 193, 2012.
Vrandečić and Krötzsch, "Wikidata: a free collaborative knowledgebase", Communications of the ACM, 2014.
Speer, Chin and Havasi, "ConceptNet 5.5: An Open Multilingual Graph of General Knowledge", AAAI 2017.