Global Triggers for Attacking and Analyzing Ranking Models

Download statistics - Document (COUNTER):

Wang, Yumeng: Global Triggers for Attacking and Analyzing Ranking Models. Hannover : Gottfried Wilhelm Leibniz Universität Hannover, Institut für Verteilte Systeme, Master Thesis, 2022, VII, 70 S. DOI:

Selected time period:


Sum total of downloads: 150

Text ranking models based on BERT are now well established for a wide range of pas-sage and document ranking tasks. However, the robustness of BERT-based rankingmodels under adversarial attack is under-explored. In this work, we argue that BERT-rankers are vulnerable to adversarial attacks targeting retrieved documents given aquery.We propose algorithms for generating adversarial perturbation of documents locallyto individual queries or globally across the dataset using gradient-based optimizationmethods. The aim of our algorithms is to add a small number of tokens to a highlyrelevant or non-relevant document to cause a significant rank demotion or promotion.Our experiments show that a few number of tokens can already change the documentrank by a large margin. Besides, we find that BERT-rankers heavily rely on the docu-ment start/head for relevance prediction, making the initial part of the document moresusceptible to adversarial attacks.More interestingly, our statistical analysis finds a small set of recurring adversar-ial tokens that when concatenated to documents result in successful rank demo-tion/promotion of any relevant/non-relevant document respectively. Finally, our ad-versarial tokens also show particular topic preferences within and across datasets,exposing potential biases from BERT pre-training or downstream datasets.
License of this version: CC BY 3.0 DE
Document Type: MasterThesis
Publishing status: publishedVersion
Issue Date: 2022
Appears in Collections:Fakultät für Elektrotechnik und Informatik

distribution of downloads over the selected time period:

downloads by country:

pos. country downloads
total perc.
1 image of flag of Germany Germany 68 45.33%
2 image of flag of United States United States 33 22.00%
3 image of flag of Netherlands Netherlands 7 4.67%
4 image of flag of Iran, Islamic Republic of Iran, Islamic Republic of 7 4.67%
5 image of flag of China China 7 4.67%
6 image of flag of Turkey Turkey 4 2.67%
7 image of flag of India India 3 2.00%
8 image of flag of Hong Kong Hong Kong 2 1.33%
9 image of flag of United Kingdom United Kingdom 2 1.33%
10 image of flag of Canada Canada 2 1.33%
    other countries 15 10.00%

Further download figures and rankings:


Zur Erhebung der Downloadstatistiken kommen entsprechend dem „COUNTER Code of Practice for e-Resources“ international anerkannte Regeln und Normen zur Anwendung. COUNTER ist eine internationale Non-Profit-Organisation, in der Bibliotheksverbände, Datenbankanbieter und Verlage gemeinsam an Standards zur Erhebung, Speicherung und Verarbeitung von Nutzungsdaten elektronischer Ressourcen arbeiten, welche so Objektivität und Vergleichbarkeit gewährleisten sollen. Es werden hierbei ausschließlich Zugriffe auf die entsprechenden Volltexte ausgewertet, keine Aufrufe der Website an sich.

Search the repository