Freedom of Speech on Campus: A pre-registered adversarial collaboration

We invite colleagues to participate in a collaborative research effort among scholars with conflicting viewpoints to jointly devise and conduct a quantitative survey on free speech on campus.

A recent study (Revers & Traunmüller 2020) claims to have found evidence for widespread support for restricting free speech on campus, experiences of conformity pressures, and self-censorship. The study attracted broad public interest (1, 2, 3, 4, 5) along with critical reactions from other scholars (1, 2, 3, 4, Reply). The criticism concerned issues of study design and item wording as well as the normative framing of the results. Some of the disagreement regarding the empirical findings seems to reflect differences in the researchers’ epistemic, methodological, and ideological backgrounds. This call and the proposed project are motivated by the desire to leverage this diversity of perspectives in a constructive way. 

Specifically, we are looking for scholars to join the group of skeptics who do not agree with the original study’s conclusions. In the form of a pre-registered adversarial collaboration, the group of skeptics along with the group of proponents work to consent on a questionnaire and analysis plan for a new quantitative survey that is to be conducted in the German university context (and potentially beyond).

Call for applications and further information

A Source Like Any Other? Field and Survey Experiment Evidence on How Interest Groups Shape Public Opinion

One result of political polarization is that citizens often react differently to new information, depending on whether they trust the communicator. Extensive evidence documents motivated reasoning and reliance on source cues for party communication. However, even in times of polarization and high political awareness citizens do not have strong attitudes towards all kinds of actors that engage in public communication. In particular, many low-profile interest groups are hardly known to the wider public. In a new study, recently published in the Journal of Communication, we (Jungherr, Wuttke, Mader, Schoen) argue that interest group communication follows different mechanisms than party communication which has previously been the focus of scholarly attention. Because most citizens lack crystallized attitudes towards low-profile interest groups, the persuasive effect of interest group communication is more homogenous than party communication so that interest groups are in a better position to persuade citizens with any political background.  

Treatment effects on attitudes towards TTIP

One aspect I like about this study, which has been years in the making, is how the study investigates the effects of interest group communication. First, we conduct a large-scale combined field- and survey-experiment in which individuals are randomly assigned to interest group communication. We partnered up with „Arbeitgeberverband Gesamtmetall“ who at the same time was interested in the efficacy of advocating for the controversial free trade agreement TTIP. Whereas one-third of respondents received no communication, the remainder were exposed to a letter that argued for the interest group’s position on this issue. Interestingly, half of the individuals received the treatment embedded in a survey-experimental which is common practice in the social sciences and allows researchers to closely examine treatment effects in a controlled setting. Yet, the other half received the letter via postal mail without any reference to our study, allowing us the investigate the persuasive effects in a realistic setting. We saw that the interest group communication succeeded in changing minds on TTIP. For every 13 letters the interest group had sent out, one person was converted from opponent to supporter of TTIP. Notably, prior attitudes towards the interest group played no discernible role in how citizens reacted to the letter, neither in the field-experimental setting nor in the more controlled survey experiment. 

Two additional studies extend this evidence to strengthen the finding that interest group communication works independently of prior credibility assessments of the source. Study 2 presents evidence in line with our pre-registered hunch that citizens have much weaker predispositions towards interest groups compared to political parties. Study 3 then puts our idea of differential effects across types of senders to a formal test in a pre-registered, highly powered survey experiment. We exposed 8,500 survey respondents to a communicative treatment where the sender cue was randomly varied. As expected, when the communication was ostensibly from a political party, the treatments only had a persuasive effect among respondents with favorable predispositions towards that party – whereas the treatment even backfired among respondents with negative attitudes towards the party. When the communication was from an interest group, prior attitudes made no difference. 

Feel free to browse the study or use our reproduction material to build on our work and let us know if you find any errors we made in the analysis. 

Symposium on ‚Opening Political Science‘

Logo of new Symposium on "Opening Political Science", published in PS: Political Science & PoliticsWhat is Open Science? Why is openness an epistemological value? Does political science need to become more open and how can we put Open Science principles into practice, considering the diversity of the methods used and questions pursued in our discipline? 

PS: Political Science & Politics has now published a symposium that gives answers to these questions and tries to push forward our collective discussion on how we as political scientists can promote openness in our work. In line with our understanding of Open Science as consisting of two principles – inclusiveness and transparency -, we as editors (Eike Rinke, Nate Breznau, Alexander Wuttke) sought to collate a symposium that reflects the breadth of political science research and the complexity of the discussions on Open Science we are having in our discipline.

This twitter thread by Eike Rinke succinctly summarizes the contributed assembled in the symposium:

Dr. Wuttke

It’s done. 

A few days after my defense, the second paper of my dissertation was published at Politics and the Life Sciences. Because the following visualization, which summarizes the paper’s argument, has never made it into the final study, I will use this blog to suppress the feeling that making the plot was a waste of time:

Are citizens turning away from democracy?

Democracy is under stress. Over the past ten years, there were more nations moving towards autocracy than countries moving towards democracy. In the past year alone, we have lost eight democracies (including Hungary, according to V-DEM data). In short, as the world is gradually is becoming less democratic we are in the midst of the third wave of autocratization (Lührmann et al. 2019). 

Some scholars argue that democracy’s eroding societal foundations are one reason for the global trend of democratic backsliding. Because citizens have grown tired of democracy political entrepreneurs and wanna-be autocrats have easy game in transforming a democratic country towards a more autocratic regime type. In two studies, we weigh the empirical evidence for the ‚democratic deconsolidation hypothesis‘.

Study 1: Have Europeans Grown Tired of Democracy? 

Study 1 [Open Access] is crossnational, examining the trajectories of democratic support among European citizens in eighteen consolidated democracies from 1981 to 2018. Because the question of democratic support is contested and scholars have previously argued that some of the presented evidence was cherry-picked we pre-registered our prospective data analyses including indicator selection and analysis syntax. Moreover, we present the entire evidence in an interactive Shiny Web Appendix. As a special treat, we use statistical techniques that allow disentangling age, cohort, and period effects. In short, our analyses do not provide much evidence for widespread democratic deconsolidation. Citizens of Europe continue to support democracy (but there is, at the same time, some evidence for an indeterminate but growing openness for trying out alternative non-democratic forms of government.). 

Abstract – British Journal of Political Science

Study 2: Examining the democratic deconsolidation hypothesis on the mass level in East and West Germany

Study 2 [Open Access] is a case study with focus on Germany that can be seen as a continuation of the cross-national analyses. Due to data availability Study 1 could only investigate attitudes towards democracy as a generic concept without considering whether citizens still support core principles of the liberal variant of democracy. Again, there is not much evidence in favor of the democratic deconsolidation hypothesis. 

Altogether, the studies can be seen as adding nuance to the very gloomy picture on the state of democratic support as democratic support is largely stable, at least in Europe. However, this is no reason to stop worrying about democracy as there is other evidence that many citizens cannot be counted on to defend democracy when it really mattes. More on this in a twitter thread.   

Abstract – Research & Politics

If you live in a Lockdown-affected country and really have no idea what to do with your time, I can offer you this 40 min video in which I run through the studies in greater detail:

When the Whole Is Greater than the Sum of Its Parts

In a new study, now published with the American Political Science Review, we show how established operationalization strategies of multi-dimensional concepts can systematically lead to wrong conclusions. Focusing on populist attitudes, we demonstrate simple methods to align theory and measurement.

Our argument refers to a specific but common type of multi-dimensional concepts which are sometimes called ‘non-compensatory’. Multi-dimensional concepts are non-compensatory when higher values on one component cannot offset lower values on another.

Think of democracy: If we believe that a country only counts as a democracy if it provides both rule of law and free elections then no valid measure of democracy will assign high democracy scores when ‚rule of law‘ scores are low even when the elections are extraordinarily fair.

However, not all social science studies consider this seemingly minor but consequential issue when putting multi-dimensional concepts into empirical practice. Focusing on populist attitudes, we show that these measurement-concept inconsistencies can lead to wrong conclusions.

Populism is an essentially contested concept. Yet, most scholars now agree that populist attitudes are multi-dimensional (e.g.: anti-elitism + Manicheanism + Sovereignty). Importantly, there is also widespread agreement on the idea that populist attitudes lie at the intersection of the concept’s sub-dimensions. Hence, the unique property of populist attitudes is the co-existence of its components. Put differently, we only consider citizens (or leaders) as populists if they accept anti-elitist views AND a Manichean outlook AND support popular sovereignty. Importantly, it is this non-compensatory concept property that distinguishes populist attitudes from other established public opinion constructs (eg cynicism, efficacy, ethnocentrism), that makes populist attitudes worthwhile as a concept and that makes it more than the sum of its parts. Yet, existing studies on populism at the mass level rarely transfer this crucial concept feature into empirical practice. Hence, some populism studies do not measure what they intend to measure and reported results do not necessarily speak about the concept under investigation.

The Causal and the Ontological Perspectives on the Relationship Between Concept and Concept Attributes

We argue that the most-often used operationalization approach (CFA or average scores) is rooted in a measurement paradigm that is often applied to latent constructs and which implicitly views the relationship between concept and concept components as causal (‘Bollen approach’).  We argue that a different perspective is needed when multi-dimensional concepts are non-compensatory. This ontological perspective we advocate allows for two operationalization strategies that account for this concept property, namely the Sartori and the Goertz approaches.

A straightforward Goertz-procedure is to use the minimum value of the concept components. The Sartori-approach entails setting thresholds on each concept components. Both approaches ensure to only assign high values to individuals if they score high on ALL concept components. These distinctions may seem like nitpicking. But they can make a crucial difference for substantive conclusions concerning nature and correlates of populist attitudes.

Correlations Between Goertz and Bollen Concept Structures of Populist Attitudes

Our preferred operationalization approach (Goertz) and the established approach (Bollen) result in different populism scores. Correlations between them are 0.4 to 0.9 Note: these scores were derived from the same data-generating process. They only differ in the aggregation rule!

Consider institutional trust. Bivariate Correlations with Institutional TrustBollen composite scores suggest that higher levels of populist attitudes go along with lower levels of trust. Yet, apparently, this association is driven entirely by anti-elitist orientations and not by the distinct concept of populist attitudes as an attitudinal syndrome at the intersection of ALL subdimensions. When operationalizing populist attitudes in a way that accounts for the non-compensatory relationship of the subdimensions (Goertz), then the seeming association between institutional trust and populist attitudes disappears.

Screenshot Shiny Web Application

We examined a large number of correlations in many datasets with different populism scales. In most cases, Goertz index indicates weaker associations with substantive variables than Bollen. In many cases, the conclusions do not differ. But too often they do,sometimes drastically. Using the Shiny Web Application, you can examine yourself how the disparities of the Sartori, Bollen and Goertz concept structures vary with scales and countries. Also, select a country and scale of your choice and see correlations with variables of interest. In addition, we use the Shiny App to demonstrate the relevance of researcher discretion. Specifically, using the Sartori approach we show how the estimated share of ‘populists’ various with specific details of the operationalization.

In addition to the Shiny Web Application, the study provides extensive Supplementary Materials. If your research deals with (non-compensatory) multi-dimensional concepts you might want to have a look at our step-by-step guide in Supplement 2. If you study populist attitudes (using the Schulz et al. Akkerman et al., Castanho Silva et al., Oliver/Rahn scale or the CSES scale of populist attitudes) you might want to have a look at Supplement 6 in which we discuss each scale and suitable operationalization strategies.

StudyShiny Web ApplicationReplication MaterialsSupplementary Materials