The Effects of Social Versus Economic Ideology Similarity Information on Explicit and Implicit Political Person Perception

Preregistered, Primary Hypotheses Tested in Studies 1a and 1b

In addressing these questions, we tested a series of competing hypotheses. Those labeled a are derived from the social primacy hypothesis (Crawford et al., 2017). Those labeled b are drawn from past research showing that both economic and social ideology individuating information affect the extent to which perceiver ideology predicts attitudes and perceived value violation in political person perception (Czarnek et al., 2019). Hypotheses labeled c are consistent with research showing that stigmas resulting from controllable group memberships are stronger than those resulting from uncontrollable group memberships (e.g., Weiner et al., 1988) and thus may be more difficult to eliminate.

Preregistered Secondary Hypotheses Tested in Studies 1a and 1b

We also tested a series of preregistered, secondary hypotheses in Studies 1a and 1b.

Non-Preregistered Secondary Hypotheses Tested in Studies 1a and 1b

Some secondary hypotheses tested in Studies 1a and 1b were not preregistered. These were (labels of a and b are unrelated to previous such labels):

Experimental Design

The experimental design for both studies was a 3 (information: social category vs. social ideology similarity vs. economic ideology similarity) × 2 (target: Democrat vs. Republican) mixed-model design. Target was the within-subjects factor.

Study 1a and 1b Participants

An a priori power analysis revealed that, in Studies 1a and 1b, a sample size of N = 111 was required to detect the effect of interest. We intentionally over-collected data by threefold due to preregistered data discard criteria, and at that point, we stopped data collection. We recruited participants via Prime Panels for all studies. Sample characteristics can be seen in Table 1.

Studies 1a and 1b initially consisted of 361 self-categorized Democrats and 367 self-categorized Republicans, respectively (see Supplementary Materials for details of self-categorization). After performing preregistered data exclusions, Studies 1a and 1b consisted of 177 Democrats and 192 Republicans, respectively. It should be noted that among these exclusions were Democratic participants in the social category information condition who self-identified as having a conservative overall political ideology, and Democratic participants in the social and economic ideology similarity conditions who self-identified as having conservative social and economic ideologies, respectively. The same was true for Republican participants who had these three types of liberal ideologies in these same experimental conditions¹.

Data were analyzed including and excluding outliers and suspicious participants. Suspicious participants were retained in both studies because their exclusion did not affect the pattern of results. However, outliers (responses > ± 2.5 SDs) affected the results of several analyses across Studies 1a and 1b, so they were excluded from all reported analyses for consistency (see Supplementary Materials for results of analyses where their exclusion changed patterns of significance and for effects of discarding inattentive participants).

Stimuli and Measures

Procedure

The procedure was identical in both studies. We embedded attention and manipulation checks throughout the survey, as well as suspicion probes. After providing informed consent, participants were instructed to memorize the full names of and information about two individuals. These individuals were the Democratic and Republican targets whose pictures were shown with their full names and the descriptions of them for that particular information condition (social category, social ideology similarity, or economic ideology similarity). Participants then were asked questions about the content of the information that had been presented. If they answered incorrectly, they were allowed up to two additional attempts before their data would ultimately be discarded (though these participants were allowed to complete the survey). They then completed demographic items and a second set of manipulation checks about the targets to ensure they had retained the information and to further reinforce the manipulation. Participants were once again allowed up to three total attempts to answer the questions correctly. After answering these questions, they completed the dependent measures, followed by the suspicion probes, and finally were debriefed.

Preliminary Data Transformations and Reporting Strategy

In preparation for our analyses, we created difference scores for all within-subjects measures². All difference scores were scored such that higher scores reflected more negative sentiment toward or worldview conflict with the political outgroup (i.e., anti-Republican in Study 1a, anti-Democratic in Study 1b). Raw cell means (i.e., non-difference scores) are reported in Supplementary Materials.

The preliminary tests of our hypotheses were the omnibus F-tests in the one-way ANOVAs (information: social category information vs. social ideology similarity information vs. economic ideology similarity information) performed on the difference scores. All were significant unless otherwise noted, and all difference scores showed anti-Republican bias in all three experimental conditions. These analyses and the omnibus ANOVAs are reported in Supplementary Materials because the more precise tests of our hypotheses were the pairwise comparisons (using Tukey’s HSD) discussed below. Although Tukey’s HSD already corrects for familywise error, we set an even more conservative standard by adjusting alpha for Tukey’s HSD to .0167 in accordance with Bonferroni’s correction.

Affective Polarization

Worldview Conflict

Support for Moderation Hypotheses

Moderation results are reported in Supplementary Materials to maintain focus on the most informative results from the present research because moderation results were inconsistent and, in some cases, counter to theoretical expectations. The hypotheses mostly were not supported.

Affective Polarization

Worldview Conflict

Support for Moderation Hypotheses

For the same reasons as in Study 1a, moderation results are reported in Supplementary Materials.

Hypothesis

Study 2a was entirely exploratory given the current debate over the fast- vs. slow-learning nature of implicit evaluations (for a review, see Ferguson et al., 2014). However, for Study 2b, the results of Study 2a along with slow-learning perspectives on implicit evaluations (e.g., McConnell & Rydell, 2014) led us to predict that:

Support for H7 also would be consistent with aforementioned literature on controllable vs. uncontrollable stigmas (e.g., Weiner et al., 1988).

Experimental Design

Both studies employed a one-way (information: social category information vs. social ideology similarity information vs. economic ideology similarity information) between-subjects design. Although participants did review information about one Democratic and one Republican target, as they did in Studies 1a and 1b, scores on the implicit measure that was used—the Implicit Association Test (IAT; Greenwald et al., 1998)—are difference scores, so they inherently incorporate the within-subjects target factor.

Participants

The power analysis for Studies 2a and 2b was identical to that for Studies 1a and 1b because both studies employed one-way between-subjects analyses with three levels performed on difference scores. Thus, 111 participants were needed to detect an effect size of f = .15. We over-collected data by approximately two-fold in this study (our stop point); we anticipated discarding a large amount of data, but not as much as in Studies 1a and 1b. We again recruited participants via Prime Panels. Sample characteristics can be seen in Table 1.

Studies 2a and 2b initially consisted of 269 self-categorized Democrats and 254 self-categorized Republicans, respectively. After performing the same preregistered data exclusions as before, Studies 2a and 2b consisted of 124 Democrats and 155 Republicans, respectively. Outliers did not affect the results of either study, so they were retained to maximize sample size.

Stimuli and Measures

Stimuli for Studies 2a and 2b were identical to those used in Studies 1a and 1b, respectively. The dependent measure was scores on an IAT that measured affective polarization in implicit political person perception. The categories used in the IAT were the two individual targets’ names (Justin and Gary), Good, and Bad. Stimuli for the Justin and Gary categories were the same pictures of Justin and Gary that had been presented along with their descriptions (or cropped or greyscale versions thereof), and stimuli for the Good and Bad categories were words relevant to these two attributes (see Supplementary Materials). Participants also completed demographic items and, in Study 2b, suspicion probes³.

Procedure

The procedure for learning the ideology similarity information or social category information was identical to that employed in Studies 1a and 1b. The only procedural difference was that, instead of completing explicit measures, participants completed the IAT, which was administered using the IATGen app for Qualtrics (Carpenter et al., 2019).

Preliminary Scoring

IAT scores (D) were computed by IATGen, which uses the scoring algorithm suggested by Greenwald et al. (2003). In this study, positive D scores indicated stronger implicit associations between the Democratic target and good words and the Republican target and bad words relative to the reverse pairing of these concepts.

Implicit Affective Polarization

Single-sample t-tests revealed bias against the Republican target in all three information conditions, M_{SocialCategoryInformation} = 0.20, SD = 0.49, t_{SocialCategoryInformation}(42) = 2.63, p = .012⁴, M_{SocialIdeologyInformation} = 0.21, SD = 0.48, t_{SocialIdeologyInformation}(35) = 2.69, p = .011, M_{EconomicIdeologyInformation} = 0.25, SD = 0.39, t_{EconomicIdeologyInformation}(44) = 2.63, p < .001. A one-way between-subjects ANOVA revealed that ideology similarity information did not shift this bias, F(2, 123) = 0.16, p = .86, ${\mathrm{\eta }}_{\mathrm{p}}^2$ < .01, BF₁₀ = 0.09⁵. Thus, information suggesting that the Republican target had a similar social or economic ideology as did participants likely had no effect on implicit affective polarization, and the Bayes Factor suggested that the evidence in favor of the null hypothesis was strong.

Discrepancies Among Measures of Affective Polarization

Discrepancies emerged in the results obtained from the two measures of affective polarization (feeling thermometer and liking scale). Although we lack data that can directly explain this discrepancy, past literature suggests that there may be issues with the reliability and validity of Likert-type measures of attitudes in political prejudice research. For instance, Price‐Blackshear et al. (2019) found that an electability Likert-type scale and a semantic differential (i.e., different Likert-type measures of similar constructs) yielded different results within the same sample. In addition, these authors also found that the same semantic differential measure resulted in different findings when administered in two different samples drawn from the same population despite highly similar research designs and identical stimulus materials.

Moreover, prior work has shown inconsistencies in results between different measures of attitudes and prejudice, such as Likert-type and feeling thermometer measures. Consistent with the results of the present research, West and Iyengar (2022) found more evidence that affective polarization was reduced when using a feeling thermometer measure compared to a Likert-type measure. In addition, Crawford et al. (2017) found different levels of support for the social primacy hypothesis when using different measures of explicit prejudice (e.g., feeling thermometer, social distance measure).

Because past results suggest reliability and validity issues with the Likert-type measure of affective polarization, in the context of the present research, we consider the evidence from our feeling thermometer (which showed that social and economic ideology similarity information reduced affective polarization to an equal extent) to warrant more confidence than those from the liking scale (which revealed that neither type of ideology information reduced affective polarization). The discrepancy is likely not attributable to sampling error given that it appeared in both samples and that it parallels results from the aforementioned previous research. Indeed, feeling thermometers have been demonstrated to be valid measures of affective polarization in multi-party systems (e.g., Gidron et al., 2022) and have been shown to be reliable measures of prejudice (for a review, see Lolliot et al., 2015).

While the IAT also showed no effects of counterstereotypic ideology information on affective polarization, the IAT should not be classified together with the two explicit measures in considering patterns of findings. This is because multiple theoretical perspectives on implicit evaluations posit that separate cognitive processes govern implicit versus explicit evaluations (see, e.g., Gawronski & Bodenhausen, 2011; McConnell & Rydell, 2014). Instead of considering the IAT results in tandem with the results from the explicit affective polarization measures, we believe that the results from the implicit measure provide evidence that implicit political person perception is governed by slow-learning processes, which is consistent with strict dual-process models of social cognition (e.g., McConnell & Rydell, 2014), and inconsistent with other models of implicit social cognition (e.g., De Houwer, 2014; Gawronski & Bodenhausen, 2011).

Discrepancies Between Social and Economic Worldview Conflict

We found that, in both samples, results from the social worldview conflict measure supported multi-dimensional perspectives on political ideology, whereas results from the economic worldview conflict measure supported single-dimensional perspectives on political ideology. While the results therefore do not provide clear support for either perspective, we interpret the findings as lending some tentative support for the social primacy hypothesis because this suggests that social ideology similarity information exerts particularly powerful effects on domain-specific worldview conflict. Specifically, the support for single-dimensional perspectives stems from the fact that social and economic ideology similarity information reduced economic worldview conflict to an equal extent. This suggests that social ideology similarity information may be so effective a means of reducing domain-specific worldview conflict that it reduces economic worldview conflict just as much as economic ideology similarity information does. The reverse was not true—economic ideology similarity information reduced social worldview conflict less than did social ideology similarity information, suggesting that social similarity information may have more powerful effects than economic ideology information on domain-specific worldview conflict. This is broadly consistent with the social primacy hypothesis.

Discrepant Results From Democratic vs. Republican Samples

While the results of the ANOVAs were consistent between Democratic and Republican samples, there were several inconsistencies between the two samples in the moderation results. Specifically, 20 moderation analyses were performed in each of these studies, and significance patterns for these analyses differed in seven of these pairs of analyses (see Supplementary Materials). The direction of these inconsistencies did not follow any particular pattern. It should be noted that providing similarity information like that in the present research is akin to providing individuating information in that it suggests that the individuals diverge from the social category stereotype, and it is not unprecedented in the political psychology literature to find discrepancies between how Democrats and Republicans are affected by individuation processes (see Price‐Blackshear et al., 2019; West & Iyengar, 2022). It is possible that these differences between Democrats and Republicans in individuation processes (including the results of the moderation analyses in the present research) are attributable to differences between Democrats and Republicans in cognitive style such as rigidity (e.g., Jost et al., 2003; Jost, 2006). However, given the unexpected direction of some of the moderation results, we invite future work to replicate the present research in service of better understanding potential differences between Democrats and Republicans.

The Social Primacy Hypothesis

The present research tested predictions relevant to the social primacy hypothesis (Crawford et al., 2017), which posits that conflict in social ideology contributes more strongly to affective polarization than does conflict in economic ideology. Consistent with previous research (Czarnek et al., 2019), we found evidence that social and economic conflict equally contribute to affective polarization based on our results from our overall worldview conflict measure and the feeling thermometer (but not the liking scale); thus, our data did not support the social primacy hypothesis (though it provided only an indirect test of this hypothesis). Additional previous research has found only mixed support for the social primacy hypothesis (Crawford et al., 2017). Although the Likert-type measure of liking did not show evidence of shifts in affective polarization, due to the aforementioned potential psychometric issues with this measure, we believe that more credence should be lent to the results of the feeling thermometer measure.

Multi- vs. Unidimensional Accounts of Political Ideology

We found support for hypotheses predicting equal effects of social and economic ideology similarity information on affective polarization and overall worldview conflict. This lent some limited support for unidimensional accounts of political ideology (e.g., Jost, 2006; Jost et al., 2003)—though these accounts were not directly tested by those particular measures—in that dimensions conceptualized to be separate (social and economic ideology) by multi-dimensional accounts of ideology exerted mostly equal effects on affective polarization and overall worldview conflict, which shows that they may operate similarly in these domains. Evidence from the domain-specific worldview conflict measures did not provide a clear pattern of support for either perspective since the social worldview conflict measure showed support for multi-dimensional models and the economic worldview conflict measure showed support for single-dimensional perspectives.

Fast- vs. Slow-Learning Accounts of Implicit Evaluations

The finding that implicit evaluations did not shift in the presence of counterstereotypic individuating information lends support to slow-learning accounts of implicit evaluations (e.g., McConnell & Rydell, 2014). These results stand in contrast to other research that has examined the effects of individuating information on implicit judgments of individual members of social groups, which has found that implicit stereotypes in person perception are subject to fast-learning (e.g., Rubinstein & Jussim, 2019; Rubinstein et al., 2018) or at least are partially or circumstantially sensitive to valid social information (Cao & Banaji, 2016; Rubinstein et al., 2021) and thus supported fast-learning theories of implicit evaluations (e.g., De Houwer, 2014).

One potential reason for this discrepancy may be the perceived controllability of political party membership; when members of a group are perceived as having a choice in their group membership, stigma and prejudice against these groups tend to be stronger (e.g., Weiner et al., 1988). Thus, it may also be harder to shift prejudices that are based on group memberships that are perceived to be controllable. Previous research showing sensitivity of implicit judgments in person perception to valid social information used targets belonging to groups (race and gender groups) in which membership arguably may be seen as less controllable than in political groups.