Open research

Datasheets, Data Statements and Model Cards: Documenting NLP Data and Models

Open research · Published 2026-10-26

An old bound laboratory logbook lying open on a desk, its handwritten pages yellowed and stained
A laboratory logbook on public display at Lawrence Berkeley National Laboratory. Photo: Daderot, public domain.

Electronic components come with datasheets listing their operating conditions and limits. In 2018 several research groups argued that datasets and machine learning models deserved the same treatment. The formats they proposed, datasheets, data statements and model cards, are now standard practice and, in part, legal expectations. Here is what each contains and how to write one that people will actually use.

Why documentation matters in NLP

Language data is never neutral. A corpus reflects who wrote the texts, where, when and for whom, and models trained on it reproduce those patterns. Without documentation, a team reusing a dataset may not know that its "English" is mostly American news from one decade, that annotators shared a particular background, or that the licence forbids commercial use. Documentation is also the backbone of reproducibility: a result cannot be checked if no one knows exactly what data it came from (see our guide to reproducibility in NLP).

Datasheets for datasets

Timnit Gebru and colleagues first circulated "Datasheets for Datasets" as a preprint in March 2018; it was published in Communications of the ACM in December 2021. Rather than a fixed template, it offers questions grouped by the stages of a dataset's life:

The authors stress that the process of answering the questions, ideally before collecting data, is as valuable as the final document, because it forces creators to think about consequences.

Data statements for NLP

Emily M. Bender and Batya Friedman proposed data statements specifically for language data in "Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science" (TACL 2018). Their key argument is that a model generalises well only to language similar to its training data, so users need to know what that language was. A data statement covers:

Bender, Friedman and Angelina McMillan-Major later published a practical guide with a short and a long form. Data statements are particularly useful for detecting mismatches that lead to bias, as discussed in our article on bias in NLP.

Model cards

Margaret Mitchell and colleagues presented "Model Cards for Model Reporting" at the FAT* conference (now FAccT) in January 2019. A model card is a short document accompanying a trained model, with sections for model details, intended uses and out-of-scope uses, relevant factors (groups, environments, instruments), metrics, evaluation and training data, quantitative analyses, ethical considerations, and caveats. Its most distinctive recommendation is disaggregated evaluation: reporting performance separately for relevant groups and conditions, such as dialects, age groups or input domains, rather than a single average that can hide poor performance for some users.

Model cards have been widely adopted by model-hosting platforms, which display them as the landing page of each model. Quality varies widely: many cards are brief or copied from templates. A useful card states concrete limitations and evaluation results that a reader could check.

Related formats

Other proposals build on these ideas: Google researchers' "Data Cards" (Pushkarna, Zaldivar and Kjartansson, FAccT 2022) focus on summaries readable by non-specialists; "system cards" describe whole products built from several models. Whatever the format, the content matters more than the name: provenance, composition, intended use, evaluation and limits.

The legal angle in the EU

The EU AI Act, Regulation (EU) 2024/1689, turns some of this documentation into obligations. Since 2 August 2025, providers of general-purpose AI models must keep technical documentation (Annex XI lists its content, including information on training data and evaluation), provide information to developers who build on the model (Annex XII), and publish a "sufficiently detailed summary" of the content used for training (Article 53). The Commission's AI Office published a template for that public summary in July 2025. Providers of models released under free and open-source licences are exempt from the first two duties unless their model presents systemic risk, but not from the training-content summary; see our guide to open language models.

Writing documentation that helps

  1. Start before collecting data, using the datasheet questions as a design checklist.
  2. Be specific. "Diverse web text" says nothing; "English-language news articles from these 12 outlets, 2015–2020, deduplicated" says a lot.
  3. State what is unknown. If speaker demographics are not available, say so rather than leaving the field empty.
  4. Record the licence of every component (see dataset licences) and how personal data was handled.
  5. Report disaggregated results for the groups and conditions that matter to users.
  6. Version the document with the dataset or model, and update it when errors are found.

Sources

Read next

More in Open research
Two handwritten pages of a nineteenth-century laboratory notebook with a small sketch of apparatus and dated entriesOpen research

Reproducibility in NLP

Preprocessing, seeds, versions and silent API updates: why NLP results fail to repeat, what studies found, and a routine that makes your work checkable.

Rows upon rows of stacked boxes of punched record cards filling a large warehouse floorOpen research

Dataset licences for NLP

CC BY, ShareAlike, NonCommercial, ODbL, LDC agreements and the EU's text and data mining exceptions, explained for people who build and share corpora.

Two women programmers set up the wiring panels of the room-sized ENIAC computer in a black-and-white photographEthics & data

Bias in NLP

Stereotypes in embeddings, gendered mistranslations, dialects flagged as offensive: what research has shown about bias in language technology and what helps.