Citation needed: A taxonomy and algorithmic assessment of Wikipedia's verifiability

Download statistics - Document (COUNTER):

Redi, M.; Morgan, J.; Fetahu, B.; Taraborelli, D.: Citation needed: A taxonomy and algorithmic assessment of Wikipedia's verifiability. In: The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019, S. 1567-1578. DOI:

Repository version

To cite the version in the repository, please use this identifier:

Selected time period:


Sum total of downloads: 161

Wikipedia is playing an increasingly central role on the web, and the policies its contributors follow when sourcing and fact-checking content affect million of readers. Among these core guiding principles, verifiability policies have a particularly important role. Verifiability requires that information included in a Wikipedia article be corroborated against reliable secondary sources. Because of the manual labor needed to curate Wikipedia at scale, however, its contents do not always evenly comply with these policies. Citations (i.e. reference to external sources) may not conform to verifiability requirements or may be missing altogether, potentially weakening the reliability of specific topic areas of the free encyclopedia. In this paper, we aim to provide an empirical characterization of the reasons why and how Wikipedia cites external sources to comply with its own verifiability guidelines. First, we construct a taxonomy of reasons why inline citations are required, by collecting labeled data from editors of multiple Wikipedia language editions. We then crowdsource a large-scale dataset of Wikipedia sentences annotated with categories derived from this taxonomy. Finally, we design algorithmic models to determine if a statement requires a citation, and to predict the citation reason. We evaluate the accuracy of such models across different classes of Wikipedia articles of varying quality, and on external datasets of claims annotated for fact-checking purposes.
License of this version: CC BY 4.0 Unported
Document Type: BookPart
Publishing status: publishedVersion
Issue Date: 2019
Appears in Collections:Forschungszentren

distribution of downloads over the selected time period:

downloads by country:

pos. country downloads
total perc.
1 image of flag of United States United States 51 31.68%
2 image of flag of Germany Germany 50 31.06%
3 image of flag of Netherlands Netherlands 14 8.70%
4 image of flag of Czech Republic Czech Republic 7 4.35%
5 image of flag of Poland Poland 6 3.73%
6 image of flag of Ukraine Ukraine 3 1.86%
7 image of flag of Russian Federation Russian Federation 3 1.86%
8 image of flag of Europe Europe 3 1.86%
9 image of flag of China China 3 1.86%
10 image of flag of India India 2 1.24%
    other countries 19 11.80%

Further download figures and rankings:


Zur Erhebung der Downloadstatistiken kommen entsprechend dem „COUNTER Code of Practice for e-Resources“ international anerkannte Regeln und Normen zur Anwendung. COUNTER ist eine internationale Non-Profit-Organisation, in der Bibliotheksverbände, Datenbankanbieter und Verlage gemeinsam an Standards zur Erhebung, Speicherung und Verarbeitung von Nutzungsdaten elektronischer Ressourcen arbeiten, welche so Objektivität und Vergleichbarkeit gewährleisten sollen. Es werden hierbei ausschließlich Zugriffe auf die entsprechenden Volltexte ausgewertet, keine Aufrufe der Website an sich.

Search the repository