Parallel Corpora: How Translated Text Trains Machine Translation

Multilingual & low-resource · Published 2026-10-26

A printed notice taped to a glass door announcing holiday closing dates in English, with a greeting in Basque below — A bilingual English and Basque notice in Boise, Idaho. Photo via Wikimedia Commons, public domain.

The Rosetta Stone, carved in 196 BC with the same decree in hieroglyphic, Demotic and Greek scripts, let scholars decipher Egyptian hieroglyphs in the 1820s. Modern machine translation rests on the same idea at enormous scale: collections of texts paired with their translations, called parallel corpora or bitexts. This guide explains how they are built, where the major ones come from, and why their quality matters.

From documents to aligned sentences

Translations are produced document by document, but statistical and neural translation systems learn from sentence pairs. Building a parallel corpus therefore involves matching documents with their translations, splitting them into sentences, and deciding which sentences correspond. This last step is harder than it looks: translators merge, split, reorder and occasionally omit sentences.

A classic solution came from William Gale and Kenneth Church at AT&T Bell Laboratories, in "A Program for Aligning Sentences in Bilingual Corpora" (Computational Linguistics, 1993). They observed that long sentences tend to be translated by long sentences and short by short, and aligned texts using sentence length in characters with a simple probabilistic model. Later aligners added bilingual dictionaries and, more recently, multilingual sentence embeddings that map sentences with the same meaning to nearby vectors regardless of language.

Landmark collections

Parliamentary proceedings

Bilingual parliaments publish verbatim records in each language. In the late 1980s and early 1990s, IBM researchers including Peter Brown and colleagues trained the first statistical translation models on the proceedings of the Canadian Parliament (the Hansard) in English and French. In Europe, Philipp Koehn released Europarl in 2005, extracted from the proceedings of the European Parliament; first covering 11 languages, it grew with EU enlargement to include 21 and became a standard benchmark for research.

EU legislation and translation memories

EU law is published in all official languages, which makes it an exceptional multilingual resource. The European Commission's Joint Research Centre released the JRC-Acquis corpus of EU legislation in 2006. The Commission's Directorate-General for Translation has published its translation memory, DGT-TM, since November 2007, with annual updates since 2012; it covers the 24 official EU languages and may be reused free of charge under the Commission's reuse decision, with attribution. Its language is legal and administrative, which is ideal for terminology but less representative of everyday speech.

OPUS

Jörg Tiedemann at the University of Helsinki started OPUS, an open collection that gathers, aligns and redistributes parallel data from many sources, from software localisation files to film subtitles and religious texts. Described in a 2012 paper at the Language Resources and Evaluation Conference (LREC), it remains one of the main entry points for researchers looking for bitext in a given language pair, with licence information for each sub-corpus.

Mining the web

Most parallel data now comes from the web. Two main strategies exist. The first finds websites that publish the same pages in several languages and aligns them; ParaCrawl, described by Marta Bañón and colleagues in "ParaCrawl: Web-Scale Acquisition of Parallel Corpora" (ACL 2020) and funded under the EU's Connecting Europe Facility, did this at scale for EU official languages paired with English. The second, "bitext mining", embeds billions of sentences from monolingual crawls in a shared multilingual space and pairs sentences that are close to each other, regardless of which site they came from; Facebook AI's WikiMatrix and CCMatrix collections were built this way.

Scale comes at a cost. Mined pairs can be partial translations, near-paraphrases, boilerplate or the output of earlier machine translation systems. Audits of crawled multilingual data (discussed in our article on low-resource languages) found that quality is worst precisely for the smaller languages where such data is most needed. Filtering tools score pairs for alignment and fluency, and research consistently shows that cleaning matters as much as quantity.

Translationese and direction

Translated text is not quite like text originally written in a language. Translators tend to simplify, to make implicit information explicit, and to carry over structures from the source; the Swedish researcher Martin Gellerstam called this "translationese" in 1986. For machine translation, this has two consequences. A parallel corpus has a direction (which side was the original), and systems may perform differently when translating original text than translated text. And test sets built by translating in one direction can flatter systems in ways that do not hold for real users. Good corpora record which side is the source.

Beyond machine translation

Parallel corpora are also used to build bilingual dictionaries and terminology databases, to project linguistic annotations from a well-resourced language onto a less-resourced one, to study translation itself, and to induce cross-lingual links between word senses, a topic explored in our article on multilingual lexical knowledge bases. Concordancers that display a word with its translations in context are everyday tools for professional translators, and the EU's own resources are part of the wider European language technology landscape.

How well a system trained on such data serves ordinary readers depends on whether their texts resemble it. Legal and parliamentary training data helps with formal documents but not necessarily with the abbreviations of a classified advertisement, as our guide on what machine translation can and cannot do for a tenant reading a lease shows. Measuring that gap is the job of translation evaluation.

Before you use a parallel corpus

Check the licence of each sub-corpus; aggregated collections mix very different terms.
Know the domain: legal, parliamentary, subtitles and web text produce very different systems.
Read samples in both languages, or ask a speaker to, before training on millions of pairs.
Remove overlaps between training data and your test set, including near-duplicates.
Record translation direction where it is known, and report it.

Sources

Gale and Church, "A Program for Aligning Sentences in Bilingual Corpora", Computational Linguistics 19(1), 1993.
Brown and colleagues, "A Statistical Approach to Machine Translation", Computational Linguistics, 1990.
Koehn, "Europarl: A Parallel Corpus for Statistical Machine Translation", MT Summit X, 2005.
Steinberger and colleagues, "The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages", LREC 2006.
European Commission, Joint Research Centre, DGT-Translation Memory, page consulted October 2026.
Tiedemann, "Parallel Data, Tools and Interfaces in OPUS", LREC 2012.
Bañón and colleagues, "ParaCrawl: Web-Scale Acquisition of Parallel Corpora", ACL 2020.
Gellerstam, "Translationese in Swedish novels translated from English", in Translation Studies in Scandinavia, 1986.