“Are Machines Better Than Humans in Image Tagging?” - A User Study Adds to the Puzzle

Download statistics - Document (COUNTER):

Ewerth, R.; Springstein, M.; Phan-Vogtmann, L.A.; Schütze, J.: “Are Machines Better Than Humans in Image Tagging?” - A User Study Adds to the Puzzle. In: Jose, J. et al. (Eds.): Advances in Information Retrieval : 39th European Conference on IR Research, ECIR 2017, Aberdeen, UK, April 8-13, 2017, Proceedings. Cham : Springer, 2017 (Lecture Notes in Computer Science ; 10193), S. 186-198. DOI: https://doi.org/10.1007/978-3-319-56608-5_15

Repository version

To cite the version in the repository, please use this identifier: https://doi.org/10.15488/9788

Selected time period:


Sum total of downloads: 98

“Do machines perform better than humans in visual recognition tasks?” Not so long ago, this question would have been considered even somewhat provoking and the answer would have been clear: “No”. In this paper, we present a comparison of human and machine performance with respect to annotation for multimedia retrieval tasks. Going beyond recent crowdsourcing studies in this respect, we also report results of two extensive user studies. In total, 23 participants were asked to annotate more than 1000 images of a benchmark dataset, which is the most comprehensive study in the field so far. Krippendorff’s α is used to measure inter-coder agreement among several coders and the results are compared with the best machine results. The study is preceded by a summary of studies which compared human and machine performance in different visual and auditory recognition tasks. We discuss the results and derive a methodology in order to compare machine performance in multimedia annotation tasks at human level. This allows us to formally answer the question whether a recognition problem can be considered as solved. Finally, we are going to answer the initial question.
License of this version: CC BY 4.0 Unported
Document Type: bookPart
Publishing status: publishedVersion
Issue Date: 2017
Appears in Collections:Zentrale Einrichtungen

distribution of downloads over the selected time period:

downloads by country:

pos. country downloads
total perc.
1 image of flag of Germany Germany 47 47.96%
2 image of flag of No geo information available No geo information available 18 18.37%
3 image of flag of United States United States 16 16.33%
4 image of flag of France France 3 3.06%
5 image of flag of Austria Austria 3 3.06%
6 image of flag of United Kingdom United Kingdom 2 2.04%
7 image of flag of Czech Republic Czech Republic 2 2.04%
8 image of flag of China China 2 2.04%
9 image of flag of Ukraine Ukraine 1 1.02%
10 image of flag of Italy Italy 1 1.02%
    other countries 3 3.06%

Further download figures and rankings:


Zur Erhebung der Downloadstatistiken kommen entsprechend dem „COUNTER Code of Practice for e-Resources“ international anerkannte Regeln und Normen zur Anwendung. COUNTER ist eine internationale Non-Profit-Organisation, in der Bibliotheksverbände, Datenbankanbieter und Verlage gemeinsam an Standards zur Erhebung, Speicherung und Verarbeitung von Nutzungsdaten elektronischer Ressourcen arbeiten, welche so Objektivität und Vergleichbarkeit gewährleisten sollen. Es werden hierbei ausschließlich Zugriffe auf die entsprechenden Volltexte ausgewertet, keine Aufrufe der Website an sich.

Search the repository