More broadly, analyzing the conditions for policy contestation creates new opportunities for climate policy by suggesting that a focus on tempering contestation may be useful instead of the typical (but challenging) goal of securing acceptance or support for climate policy. For example, contestation over climate policy may grow in the future due to increasing ambition, challenging climate policy scholars and practitioners to consider how to respond to contestation. Finding ways to reduce contestation enough to achieve ‘sufficiently’ acceptable climate policy could be a worthwhile goal, and understanding the drivers of contestation is a stepping stone in this direction.
We use qualitative comparative analysis (QCA) to examine possible associations between our posited conditions (both individually and in combination) and the outcome of policy contestation. QCA is a set-theoretic method that assesses the co-presence of conditions and outcomes, thereby helping to identify which conditions could contribute to explaining an outcome through their association. QCA does not assess causation per se but can provide evidence to inform causal reasoning in broader argumentation. It builds on case study logic where close attention is given to understanding which combinations of conditions produce an outcome, and extends this logic to be able to study a larger number of cases. It thereby reflects a middle ground between qualitative and quantitative approaches that helps to systematically compare rich individual cases. It is useful when explanatory conditions are complex constructs requiring interpretive sensibility informed by different available sources of data, and where conjunctional causation (i.e., combinations of conditions producing an outcome) and equifinality (i.e., multiple causal pathways producing an outcome) could be important. Conditions and outcome(s) are measured by scoring their degree of presence in a case. Fuzzy-set QCA (fsQCA), which we use here, enables scoring the degree of presence of conditions/outcomes in a graduated manner (e.g., “fully in”, “partially in”, or “fully out” of a set) using consistent criteria. Overall, we employ QCA as both a research approach and a technique (following Schneider and Wagemann) by designing data collection to feed into calibration of conditions on a set theoretic basis involving multiple sources of heterogeneous data to enable an in-depth qualitative understanding of the cases, examining different possible solutions and their implications, and from the outset considering the possibility of conjunctural causation and equifinality due to potentially diverse dynamics of policy contestation.
In the following subsections we explain: (i) conceptual design focusing on policy contestation as the outcome, conditions considered, and excluded conditions; (ii) case selection; (iii) operationalization; (iv) analysis; and (v) robustness considerations.
Our outcome of interest is post-adoption policy contestation. The term “contestation” encompasses a variety of disputes over policy which could range from disagreement over certain features, to outright mobilization against it. We distinguish contestation from “opposition” as the latter often suggests a focus on organized activity (such as party politics and coalitions), whereas the former leaves open the degree and form of negative response among mass publics.
By contestation, we refer to negative responses to adopted climate policy among mass publics (and possibly also other actors) involving distinctly negative sentiment that may or may not be coupled with mobilization against the policy. Scholars of public responses to climate policy discuss how public responses to a policy can vary across a spectrum from support to opposition. In the middle of this spectrum is a mixed zone, where ambivalent response (e.g., somewhat positive or somewhat negative) or perhaps even an absence of clear response (e.g., due to lack of awareness) can occur. For example, PytlikZillig et al. refer to a “zone of policy tolerance” where policies may be explicitly or implicitly tolerated, even if they are not necessarily viewed favorably. Anisimova and Patterson categorize policy responses among mass publics into four types (backlash, non-acceptance, acceptance, and support). Looking across this spectrum, we demarcate those involving contestation from those which are (broadly) non-contested (Fig. 2). This also enables parsing empirical responses into a fuzzy set outcome score by giving each category a qualitative code corresponding to the degree of contestation (also see Case selection).
We identify a variety of possible socio-political conditions that could be associated with post-adoption policy contestation. This includes both policy content and context-related conditions, because it is important to consider not only what is contested (i.e., policy content) but also the context within which contestation occurs. Context-related conditions involve the characteristics of a broader policy environment that might condition policy contestation. We specifically consider: (i) socio-political polarization, and (ii) climate change concern. Policy content conditions involve policy design features influencing the costs or other burdens associated with a policy as perceived by the mass public. We specifically consider: (iii) policy demandingness, (iv) policy unfairness, and (v) information provision. These conditions are summarized in Table 5 and further explained individually next. We considered but excluded several other conditions as also explained next. Importantly, climate change concern and information provision could have a directionality opposite to the other conditions. This reflects the conceptual logic of QCA, which emphasizes that the presence and absence of a condition is not necessarily inverse, and empirically, it is more meaningful to measure the presence of these conditions than their absence.
Decision-making in a polarized society may be prone to contestation as new policies become subject to pre-existing oppositional divides, producing opposition. Polarization can be related to political beliefs and ideologies (ideological polarization) or hostility and negative emotions towards political opponents (affective polarization), which can overlap in complex ways. Polarization can impact policy making through undermining the ability to generate and sustain agreement. This could persist at the post-adoption stage by, for example, mobilizing publics through negative emotions or undermining trust in government decisions. Climate policy scholars argue that mass public views can be shaped by elite and ideological polarization, and political party stances more generally. We make a simplifying assumption to treat socio-polarization polarization as a single condition because of difficulties distinguishing between ideological and affective polarization across all countries in our sample, and within available data (e.g., interviews) especially within heated policy debates. Further disaggregation of polarization could be a next step if this condition is revealed as important in our empirical analysis.
Concern over climate change among mass public may reduce the likelihood of policy contestation due to increased willingness among people to support policy actions taken, even when these entail costs or other burdens. Lack of concern over climate change could, therefore, have the opposite effect, increasing the likelihood of policy contestation. Concern over climate change is generally thought to increase the likelihood of climate policy support and acceptance, even though support for specific policies may differ. Moreover, support/acceptance is heterogeneous within a society. Climate change concerns can involve various aspects (e.g., belief in anthropogenic climate change, risk perception, and expectation of consequences, attribution of responsibility). Here, we focus on the relative prioritization of climate change concern within a society among other policy concerns. We assume that if climate change is of high relative public concern, then the likelihood of contestation towards hard climate policies will be lower because people are more likely to believe that climate policy measures are needed.
High policy costs or other burdens on policy target groups may increase the potential for policy contestation by triggering specific criticisms among those affected and/or wider audiences. For example, this can include material/financial costs, expectations for behavior change, or symbolic costs linked to people’s identities and values. In other words, the more demanding policy expectations are for people, as they experience and perceive a policy, the more likely policy contestation may be. Issues of policy demandingness are often thought to be a central factor influencing the acceptability of climate and environmental policy measures. By policy demandingness, we refer to the extent to which a policy makes demands on target groups. Often this is thought about in terms of policy stringency, which refers to “the exact degree of ambitiousness underlying the government’s efforts to address a given issue”. For climate policy, stringency involves “the degree to which climate actions and policies incentivise or enable GHG emissions mitigation at home or abroad” or “the cost” imposed on polluting or other environmentally harmful activity. Thus, stringency usually focuses on costs/burdens in terms of the strictness of rules and requirements specified by a policy. Policy demandingness, on the other hand, focuses on the overall demand on target groups (through imposing burdens, effort, and compliance requirements). A seemingly high cost (stringency) might not always be experienced as demanding, and vice versa (as a policy that might not seem burdensome might nonetheless be perceived as such by target groups). Thus, policy demandingness needs to be assessed interpretively, considering both policy stringency (i.e., objective rules/requirements — for which we draw on the OECD Climate Actions and Policies Measurement Framework) and perceptions or experiences of target groups through interviews and secondary materials (e.g., media articles, other secondary materials).
Policies perceived as unfair may increase policy contestation due to moral judgements among mass publics that policy is unbalanced or mistargeted in society. Fairness is widely identified as a factor influencing climate policy support and acceptance and has been observed to play a key role in contestation over climate policy. In response, scholars commonly call for redistributive mechanisms to ensure fairness. Yet, others have urged caution about whether such prescriptions hold across non-western countries where beliefs about environmentalism and redistribution may be differently organized. We nonetheless focus on distributional fairness because of its prominence in climate policy thinking and because it provides a basis for cross-case comparison, while also considering perceptions of fairness through interpreting multiple sources of data (e.g., interviews, media articles, policy documents). We refer to “unfairness” (as a negative condition) given our interest in understanding conditions of policy contestation.
Information provision for a policy may decrease policy contestation by increasing understanding among mass publics, but it could also increase policy contestation by increasing policy salience, which could make it a target of criticism (e.g., under the simultaneous condition of socio-political polarization). The importance of information provision for policy acceptance is often asserted in reviews and surveys of climate policy acceptance. However, Rhodes et al. show that providing information on policy effectiveness may not translate into higher support, and Hulkkonen et al. suggest that the effect of tailored information provision on policy attitude also depends on other factors (e.g., demographic attributes, perceptions of vulnerability). Moreover, Mildenberger et al. found that exposure to individualized information about a respondent’s actual climate rebate could lead to people overestimating costs rather than benefits depending on their political context. Thus, the role of information provision is unclear, requiring consideration of both policy content as well as interpretation of how this is understood by people in practice (e.g., interviews, media).
We also considered several other policy content and context-related conditions that could be associated with policy contestation, but excluded them for various reasons. In addition to substantive reasons for excluding each individual condition as explained below, a further overall need was to identify the conditions that are most salient to our study to avoid including too many conditions relative to the number of cases. This is a key issue for our specific method of QCA, but it is of course also a critical consideration in quantitative research more generally to ensure sufficient variation and avoid overfitting. Good QCA practice therefore typically involves limiting the number of conditions, with around 4-5 conditions being appropriate for our case sample size. Nonetheless, these excluded conditions also indicate important aspects that future work could further consider.
Excluded context-related conditions include inequality, overlapping economic grievances, democratic quality, trust, type of political institutions, prior contestation, and disasters. Inequality is often invoked in discussions of public discontent. However, including it here risks overlap with conditions of unfairness and demandingness, which are the more proximate ways in which inequality might be expected to translate into policy contestation (i.e., through criticisms that a policy is unfair or too demanding for certain groups). Scholars in political sociology increasingly highlight relative deprivation or decline in social status as important in producing discontent linked to inequality, yet these are difficult to comparatively measure. Overlapping economic grievances (e.g., economic crisis) could increase the propensity of policy contestation. Although this is likely to have complex links to a specific climate policy, and we may also expect to see such concerns manifest through conditions of policy demandingness and/or unfairness, thus we exclude it from this study. Democratic quality could affect opportunities or propensity for policy contestation, although the OECD countries in our sample do not vary greatly in this regard (i.e., it is a relatively homogenous background condition). Trust could reduce the propensity for policy contestation if target groups trust policymaking processes. However, it is difficult to tie trust to a specific policy process as it typically involves a broader generalized sentiment. Moreover, people (or other actors) may trust the policymaking process but still oppose specific policies, or distrust them yet accept specific policies for other reasons. Therefore, we exclude it here. Political institutions are important in comparatively explaining climate policy adoption, such as representation (proportional or majoritarian) and interest group intermediation. However, it is less clear how such institutions might affect post-adoption policy contestation. Industry or interest groups might mobilize contention into the public sphere, which could be more likely under antagonistic majoritarian systems than proportional corporatist ones, but this would probably also be case specific and contingent, and thus difficult to compare generally. Prior protests might increase the likelihood of policy contestation by lowering uncertainty and transaction costs for contestation in a new situation. Although others have considered this condition in studying local energy projects, we exclude it due to difficulties in understanding its role at a national level. Natural disasters might dampen policy contestation due to increased concern about climate change and/or demand for climate action. However, we excluded this as a condition because of difficulties comparing disasters (which could vary in type, scale, impact, and affected groups), understanding the complex ways in which disasters take on social and political meaning in specific contexts, and assessing variation in salience and timing in relation to policy processes.
Excluded policy-related conditions included visibility of costs, inclusion in policy formulation, and proximity to elections. Visibility of costs for target groups could increase the potential for policy contestation. However, this is difficult to comparatively assess across cases, even when costs are notionally visible (such as price breakdowns on bills), due to the many ways in which costs might or might not take on meaning within specific policy debates. Inclusion in policy formulation may decrease the likelihood of policy contestation by incorporating peoples’ concerns and/or increasing legitimacy. However, again, this is difficult to assess comparatively since forms of inclusion could vary between contexts and levels of governance, and is especially challenging to measure at a national level. Proximity to elections could create political opportunities for policy contestation. Although this could be important when studying strategic contentious action, we expect that post-adoption policy contestation would largely depend on when policy is adopted and/or ramped up, provoking a response, rather than on an election cycle per se. Moreover, such timing could play different roles in different situations, making comparability difficult.
Cases in our study comprise hard climate policies (i.e., pricing/taxation, regulation, and phase-outs) at the national level in OECD member countries between 2009 and 2022. We focus on these countries as our initial case population because they are industrialized countries, which arguably should be expected to take on a leadership role advancing domestic climate policy action. We examine a 14-year period since 2009 when the Copenhagen Climate Summit (COP15) occurred, which also captures a roughly equal period before and after the 2015 Paris Agreement when debates over domestic action grew especially prominent.
This analysis builds on previous work where we systematically sampled 50 hard climate policy packages across 21 OECD member countries to assess variation in public responses to hard climate policy and the prevalence of post-adoption contestation without sampling on the dependent variable. The basis for this sampling was the open access Climate Policy Database (https://climatepolicydatabase.org), which involved: (i) screening and re-categorizing policy instruments to distinguish hard policy types, (ii) selecting two important sectors: general frameworks policies, and energy and heat policies, (iii) filtering for policies that impose costs on mass publics, and (iv) consolidating closely related policies into policy packages where appropriate. Responses were categorized into four types (support, acceptance, non-acceptance, backlash) using a typology of disapproval (involving indicators of policy dis-favorability and government dissatisfaction) and mobilization (involving indicators of participation, sustained action, and specificity in focus) using indicators measured by interpretively coding systematically sampled country-specific newspaper media.
We now seek to explain the conditions under which contestation occurs. First, we recategorize the outcome values into a fuzzy set presence measure of policy contestation. Then, we further screen cases to exclude cases that de facto do not impose significant costs or constraints on target groups (as established through further analysis of policy documents and consultation with policy experts). As a result, we retain 27 policy package cases across 16 OECD member countries (Table 6).
A key first step for conducting fsQCA is measuring and calibrating the outcome and conditions. This refers to the procedure of assigning a set presence/absence score (or partial presence/absence) for each outcome and condition. In other words, the degree to which a certain condition or outcome is present in each case. For example, a case with a high level of socio-political polarization would be deemed fully (or nearly fully) in the condition set “socio-political polarization”, whereas a case with distinct but lower level of socio-political polarization would be only partially in this set, and a case with little or no distinct socio-political polarization would be deemed outside of the set. Using consistent decision criteria, conditions, and outcomes across all cases in a sample can be scored. Such decision criteria can aggregate different sources of evidence, including varying sources of evidence for different cases, into a synthetic, criteria-based scoring decision. However, it is important to avoid overclaiming the level of precision, especially when working with complex constructs, qualitative and/or mixed sources of data, and differing combinations of data for different conditions/outcomes and/or cases. Therefore, a common approach is to delineate a limited number of qualitative thresholds (anchor points) that represent points along a spectrum of fuzzy set presence. We do this following the typical convention among fsQCA scholars using four thresholds to avoid “middling” by ensuring that every fuzzy presence score is either distinctly more in or more out of the set, rather than on the fence. These thresholds (i.e., anchor points) range from 0 to 1, where: 1 = fully present, 0.67 = mostly present, 0.33 = somewhat present, and 0 = not present. This retains a qualitative disposition while also rendering complex constructs comparable across cases.
For the outcome scores, we categorize each case (Table 6) for the presence of policy contestation as: backlash = 1, non-acceptance = 0.67, acceptance = 0.33, and support = 0 (see “Conceptual design”). The prior categorization in Table 6 involved an extensive systematic media analysis drawing on multiple case-specific newspapers and validation steps. Therefore, we assume that this input provides a valid assessment which we now re-categorize here. We also pay careful attention through the expert interviews to check that our prior outcome assessments based on secondary data are appropriate, which were indeed confirmed through the interviews.
For the condition scores, we first derive anchor points for each condition based on our reasoning about how each condition would manifest if fully, partially, or not present (Table 7). We then draw on multiple sources of evidence to score conditions in each case, centering on original primary expert interviews (n = 62), supplemented by media articles, secondary datasets, academic and gray literature, and policy documents (Fig. 3). Our goal is to enable richly informed analytical judgments for each condition. In all cases, we draw on interview data as well as media articles systematically sampled in prior analysis and secondary datasets where available. When substantial gaps remain that prevent scoring of a condition in a case, we then interrogate academic and gray literature, and policy documents, to address these gaps (Fig. 3). A summary of data sources used for scoring each condition is provided in Supplementary Note 2, and an illustrative coding example is provided in Supplementary Note 3.
We conducted structured interviews with national policy experts in all the cases between June 2023 and June 2024 (n = 62, average duration 45-60 min) (Supplementary Note 4) to gather qualitatively rich, comparable, and ground-truthed insights to inform our QCA. Interview participants were selected due to their knowledge and expertise on specific policy cases and/or as having close knowledge about climate policy with each country. We identified interviewees primarily through stakeholder mapping drawing on publicly available information about policy experts (such as attendee lists at relevant policy committees and/or expert meetings, media mentions, academic references) as well as snowballing throughout the interview phase. The number of interviews per country ranged from 3-5 depending on several factors: (i) degree of relatedness of multiple policies within a case as co-learning about several policies was common for some policy packages, (ii) knowledge of individual experts who may be knowledgeable about multiple policies, (iii) level of certainty of experts about the information they provide meaning that when low level of certainty remained about some aspects we recruited further participants to address gaps, and (iv) the overall number of policies within a specific country in our sample. Interviews were based on a structured questionnaire to explore and characterize key conditions in our analysis, while also leaving room to discuss relevant issues openly regarding the overall issue of explaining public responses to climate policy (Supplementary Note 5). The interview guide contained 10 fixed questions (supported by structured and spontaneous probing) and several open questions, in a consistent order for all participants.
We interpretatively code all relevant data (both primary and secondary) for a given case using NVivo (qualitative data analysis software) using our framework of conditions (see “Conceptual design”). This provides a consistent basis for integrating different sources of data and triangulating to make synthetic judgments to score each condition (Supplementary Note 3). The scored (i.e., calibrated) data for all cases is provided in Supplementary Note 6.
We follow established fsQCA protocol, which involves several steps. First, we analyze individual necessary conditions, which are those that are required in all possible explanations of the outcome. If no conditions are found to be individually necessary, we analyze SUIN conditions. This refers to “a sufficient but unnecessary part of a factor [i.e., pathway] that is insufficient but necessary for an outcome”, or in other words, conditions (individually or in combinations) involved in producing the outcome but not enough to fully explain the outcome on their own. Second, we analyze sufficient conditions, which are those that (individually or in combination) are sufficient for the outcome. In other words, this step seeks to identify possibly multiple pathways through which the outcome is produced. We carry out fsQCA using RStudio (version 2024.04.0 + 735) and the packages “QCA” and “SetMethods”.
We apply standard parameters of fit to results for necessity and sufficiency. For necessity these parameters are: (i) inclN (inclusion for necessity) which is the extent to which a condition (or combination) is necessary for the outcome, (ii) RoN (relevance of necessity) which is the empirical relevance of a condition (or combination) through being specifically associated with the presence of the outcome (i.e., not just present in general), and (iii) CovN (coverage of necessity) which is the extent to which a condition (or combination) deemed necessary explains the outcome. For sufficiency, these parameters are: (i) inclS (sufficiency inclusion score) which is the extent to which a sufficient combination of conditions is associated with the outcome, where typically a value ≥ 0.8 may be considered good, and 0.7-0.79 acceptable, and (ii) PRI (proportional reduction in inconsistency) which is the extent to which combinations of conditions are consistently associated with the outcome, where typically a value ≥ 0.9 may be considered highly consistent, 0.8-0.89 moderate, and 0.7-0.79 weakly acceptable.
We consider several possible types of solutions (i.e., conservative, parsimonious, and intermediate solutions). The conservative solution prioritizes reliability and higher consistency scores, the parsimonious solution prioritizes simplicity but accepts lower consistency, and the intermediate solution compromises between both. While the conservative solution can be most appropriate in the first instance because it prioritizes reliability and consistency, seeking the most robust solution given available data, in prioritizing consistency (with limited data), it can still lead to complex solutions. Thus, following QCA practice, we critically reflect on the appropriateness of different possible analytical solutions to derive the most reasonable theoretically informed empirical explanation. The step-by-step results with explanation is provided in Supplementary Note 1, also with a crisp set analysis as a sensitivity test.
The robustness of the findings is important to consider (i) regarding conceptual and empirical design, and (ii) regarding the analysis approach. First, in terms of design, our five causal conditions are unlikely to be exhaustive. We considered but excluded several other conditions (see “Conceptual design”) which could nonetheless be relevant, due to difficulties of data availability or clarity over the causal link to the outcome. As QCA explicitly highlights the challenge of limited diversity inherent to comparison (i.e., where not all possible combinations of conditions are observed empirically), it requires being selective about the conditions examined. When measuring and calibrating conditions across diverse case contexts, we needed to allow for different forms and combinations of data available. We sought to mitigate this with a standardized procedure for data collection, including primary structured interviews with experts in each case designed specifically for this purpose, combined through triangulation to produce well-reasoned reflective scoring. This helped to ensure relative comparability and to enhance “case intimacy” in calibration. Our approach thus helps to foreground the qualitative core of fsQCA, including prioritizing in-depth case knowledge, contextual understanding, and theoretical grounding alongside the technique itself.
Second, in terms of analysis, there is a risk of overfitting to the data with a somewhat high number of conditions relative to the number of cases. This is an important reason why we consider different possible solutions (i.e., conservative, parsimonious, intermediate). Doing so indicates that our findings are likely to be robust because different solution types remain consistent around core elements and with little change in parameters of fit meaning that substantive interpretations do not greatly change for different model specifications (following Schneider and Wagemann). The analysis of the pricing/taxation subgroup aligns with the results for the overall sample, although the regulation and phase-out subgroup cannot be explained. Although since the solutions and parameters of fit do not change much between the whole sample and pricing/taxation subgroup 1, this also suggests that subgroup 2 may not be entirely different, since otherwise we might expect to see a skewing effect in the overall sample. Nonetheless, further analysis of these subgroups is a useful area for future research to sharpen insights into the reasons for contestation over different hard policy instruments.
