Question order effects: how robust are survey measures on political solidarities with reference to Germany and Europe?

Abstract

The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people’s home country and Europe raising questions with respect to the order of precedence. Research has shown that the order of asking questions can have a profound impact on answer behavior compromising data quality. In this study, we therefore investigated the occurrence of question order effects in a German-European context using two questions on political solidarities. For this purpose, we conducted an experiment in a German online panel (N = 874) and analyzed response behavior and effort in terms of response times. In contrast to previous research, we found no empirical evidence for question order effects impacting people’s responses. Even though there were no response time differences between the question order conditions, the first question always took longer to respond to than the remaining one. Overall, our findings indicate the robustness of questions on political solidarities against question order effects. One potential explanation is that people have comparatively strong (or crystallized) attitudes when it comes to political solidarities.