Named Entity Recognition: How NLP Systems Find People, Places and Organisations

Methods & evaluation · By the Parallel Text editorial team · Published 2026-11-12

A metal signpost on a town square with white arms pointing to Moscow, Washington, Lucca, Eisenstadt, Schongau, Győr, Princeton and other places, each with a distance in kilometres, under a blue sky with flags — A signpost on the Place de la Mairie in Colmar, Alsace: a dozen place names, each pointing at one place in the world. Photo: Hearty chortle, CC0, via Wikimedia Commons.

Read a news story and you pick out its names without effort: who acted, where, on behalf of which company. Named entity recognition (NER) is the task of making a program do the same, by finding spans of text that name something and labelling each span with a type. It is one of the oldest tasks in natural language processing and one of the most used, inside search engines, news monitoring, document redaction and the extraction of facts from scientific papers. It is also a good example of how the way a task is defined and scored shapes what "state of the art" means.

What counts as a named entity

The term comes from the Message Understanding Conferences run for US government programmes in the 1990s; the sixth edition, MUC-6 in 1995, made recognising names of people, organisations and locations, together with dates, times, sums of money and percentages, a task of its own. The definition has stayed practical rather than philosophical: a named entity is whatever the annotation guidelines of a dataset say it is.

That makes the label set the first thing to check. The CoNLL-2003 shared task, still the most cited benchmark, uses four types: PER (person), LOC (location), ORG (organisation) and MISC, a catch-all for nationalities, events, titles of works and other names. Its English data comes from Reuters news stories and its German data from the Frankfurter Rundschau. The OntoNotes 5.0 corpus, on which many off-the-shelf English models are trained, distinguishes eighteen types, splitting places into geopolitical entities (countries, cities), other locations and facilities, and adding laws, products, works of art and numerical types such as quantities and ordinals.

The same sentence can therefore be "correct" in different ways. In The Bank of England raised rates in London, a CoNLL-style annotation tags Bank of England as an organisation and London as a location; OntoNotes would tag London as a geopolitical entity. Some corpora, such as ACE 2005 and the biomedical GENIA corpus, also annotate the entity nested inside another: England within Bank of England.

From names to tags: the BIO scheme

Most systems treat NER as sequence labelling: every token receives one tag. In the widespread BIO (or IOB) scheme, B- marks the first token of an entity, I- the following tokens of the same entity and O everything outside an entity:

The    Bank   of     England  raised  rates  in  London
O      B-ORG  I-ORG  I-ORG    O       O      O   B-LOC

The scheme marks where one entity ends and the next begins, even when two names of the same type are adjacent. One limitation is built in: a flat sequence of tags cannot represent nested entities, which is why nested NER needs other formulations, such as classifying candidate spans directly.

Tokenization matters more than it seems. Modern models split words into subword pieces, so Kinshasa may become several tokens; the system must decide which piece carries the tag (usually the first) and reassemble the words afterwards. Our article on how tokenization works explains where those pieces come from.

How the methods evolved

Rules and gazetteers

Early systems combined lists of known names (gazetteers) with hand-written patterns: a capitalised word followed by Ltd is probably a company. Rules are transparent and still useful for narrow, regular formats, but they break on ambiguity (Jordan the country, the river or the person) and on any name the list does not contain.

Statistical sequence models

From the late 1990s, hidden Markov models, maximum-entropy models and above all conditional random fields (CRFs, introduced by John Lafferty, Andrew McCallum and Fernando Pereira in 2001) learned from annotated text. They relied on features designed by hand: capitalisation, prefixes and suffixes, the surrounding words, membership of a gazetteer. Most of the systems in the CoNLL-2003 shared task were of this kind.

Neural networks and pre-trained transformers

In 2016, Guillaume Lample and colleagues showed in "Neural Architectures for Named Entity Recognition" that a bidirectional LSTM with character-level word representations and a CRF output layer could match feature-engineered systems without language-specific features or gazetteers. Since 2018 the usual approach has been to fine-tune a pre-trained transformer encoder, such as BERT or RoBERTa, on the annotated data, with a classification layer that predicts a tag for each token. Large generative models can also be prompted to list the entities in a text, which is convenient when there is no training data, but their output then has to be mapped back onto exact spans, and they can return names that do not appear in the input at all.

How NER is scored, and what the scores hide

The CoNLL convention is strict: a predicted entity counts as correct only if both its boundaries and its type match the reference exactly. Precision is the share of predicted entities that are correct, recall the share of reference entities that are found, and F1 their harmonic mean. Under this rule, tagging England instead of Bank of England counts as one false positive and one false negative, worse than predicting nothing. Other schemes, going back to the MUC evaluations, give partial credit for overlapping spans or the right type with the wrong boundaries. Neither is wrong, but a paper should say which it uses. The general cautions in our guide to evaluating NLP models apply in full.

Two further problems concern the benchmark itself. Reference annotations contain errors, and re-annotation studies of the CoNLL-2003 test set have found and corrected mislabelled sentences; at the top of a leaderboard, differences between systems can be smaller than the noise in the labels. And the test set dates from 1996 news. Shuheng Liu and Alan Ritter, in "Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023?" (ACL 2023), annotated recent news with the original guidelines and tested more than twenty models trained on the old data. Models varied widely; pre-trained transformers such as RoBERTa and T5 showed no evidence of degradation, and the authors attribute most of the deterioration they did observe to the gap in time between pre-training data and test data rather than to years of reuse of the same test set.

Beyond English news

A model trained on news stories performs worse on tweets, clinical notes, historical documents or legal judgments, where names are written, abbreviated and capitalised differently. Capitalisation itself is a cue that many languages do not offer, or offer differently: German capitalises every noun, and scripts such as Arabic, Chinese or Devanagari have no capitals at all.

Annotated data is unevenly spread across languages. WikiANN (Xiaoman Pan and colleagues, ACL 2017) derived training data for 282 languages automatically from links between Wikipedia articles: broad coverage, but noisy labels. MasakhaNER (David Ifeoluwa Adelani and colleagues, TACL 2021) took the opposite route, with a community of researchers annotating news text by hand in ten African languages, among them Hausa, Igbo, Kinyarwanda, Swahili and Yorùbá; a second edition extended it to twenty languages. The trade-off between automatic coverage and careful annotation is the central question for low-resource languages in general.

What comes after recognition

Recognising that Paris is a location does not say which Paris. Entity linking, the next step in many pipelines, connects each mention to an entry in a knowledge base, often a Wikidata identifier, so that Paris in a Texas local paper and Paris in a French election story resolve to different places, and so that Deutschland, Allemagne and Germany resolve to the same one. Our article on multilingual lexical knowledge bases describes the resources involved.

NER is also the usual first step in redacting personal data from documents before they are shared or used to train models. It helps, but it is not a guarantee: names are only part of what identifies a person, and a missed name in a long document is enough to undo the effort. Our article on GDPR and text data explains why indirect identifiers matter as much as names.

Sources

Tjong Kim Sang and De Meulder, "Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition", CoNLL 2003.
Lample and colleagues, "Neural Architectures for Named Entity Recognition", NAACL 2016; Liu and Ritter, "Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023?", ACL 2023 (both linked above).
Grishman and Sundheim, "Message Understanding Conference-6: A Brief History", COLING 1996; Lafferty, McCallum and Pereira, "Conditional Random Fields", ICML 2001; Pan and colleagues, "Cross-lingual Name Tagging and Linking for 282 Languages", ACL 2017; Adelani and colleagues, "MasakhaNER: Named Entity Recognition for African Languages", TACL 2021; OntoNotes Release 5.0, Linguistic Data Consortium, 2013.