This is an open access article distributed under the terms of the Creative Commons
Attribution License (Classical proponents of liberal democracy like Locke and Mill believed that human rationality would produce a political system conducive to achieving a just society. Similar to classical economics, wherein the combination of individual self-interest and rationality in an economy free of feudal rents and undue governmental interference would most efficiently provide for society’s material needs (as if directed by an invisible hand), liberal democracy would harness citizens’ self-interest and rationality to produce fair and efficient governance. In contrast to contemporary conservatives who doubted the ability of the masses to participate intelligently in self-governance, classical liberals pointed to the rationality inherent in human nature, which guided individual decisions and would produce, in the aggregate, a government best suited to meeting the needs of all members of society. Later proponents of liberalism adopted a stronger view of human rationality, well suited to making models of liberal political and economic systems more tractable, but at the cost of psychological accuracy (Rosenberg, 1995).
Classical liberals were more aware of the limitations to human rationality. John Stuart Mill (1967, p. 226) in particular was sensitive to the ways in which humans fall short of the rational ideal: “[I]t is, when not duly guarded against, an almost irresistible tendency of the human mind to become the slave of its own hypotheses; and when it has once habituated itself to reason, feel, and conceive, under certain arbitrary conditions, at length to mistake these conditions for laws of nature.” Modern psychological research has provided ample empirical support to Mill’s conjecture on this “almost irresistible tendency”. Motivated reasoning skews our thinking to arrive at conclusions unthreatening to our sense of self (including group identities) and consonant with our preexisting beliefs and knowledge – all outside of conscious awareness (Kunda, 1990). This bias is subtle, in that while people do attempt to be rational, “[t]hey will believe undesirable evidence if they cannot refute it, but they will refute it if they can” (Kunda, 1990, p. 490). Knowledge provides a constraint on the process of motivated reasoning: awareness of facts that would contradict a desired conclusion impedes the process of motivated reasoning to arrive at that conclusion. In fact, the phenomenon of motivated reasoning may be itself produced by a partial accumulation of knowledge and attempts to seek confirmation rather than disconfirmation of one’s beliefs (Hahn & Harris, 2014, p. 87). In this view, what looks like “motivated” reasoning may be a “spiral of conviction” process rooted in ignorance of contrary knowledge, and requiring no motivation other than accuracy (Friedman, 2012).
Motivated reasoning is not merely an ailment of the less cognitively gifted. In fact, motivated reasoning may produce greater bias among those higher in both numeracy (Kahan, Peters, Dawson, & Slovic, 2017) and cognitive reflection (Kahan, 2013). Motivated reasoning may be an “adaptation suited to promoting the interest that individuals have in conveying their membership in and loyalty to affinity groups central to their personal wellbeing” (Kahan, 2013, p. 418) – and those with greater cognitive skills tend to be “better” at it. This bias may be a design feature rather than flaw of the human mind, part of a suite of adaptations that produced better collective (if not individual) decision-making during our evolutionary history (Mercier & Sperber, 2011). Whatever the origins of motivated reasoning and other cognitive biases, their presence in the thinking of policy elites is cause for concern: whether in economists blinding themselves to ideologically uncongenial evidence about the causes of recessions (Kessler, 2010), or in international relations policymakers making disastrous foreign policy decisions (Yetiv, 2013).
If motivated reasoning biases all human thinking, particularly that of the most intelligent among us, it would undermine both the Lockean faith in human rationality as a solid foundation for democratic politics and the Burkean or Platonic faith in the unique abilities of the gifted to rule. However, there is evidence that specialized training (not overall cognitive ability) can reduce or mute motivated reasoning. In one study, lawyers and judges were found to think through legal problems with less motivated reasoning than law students and laypersons (Kahan et al., 2015). Other studies have found that biases in logical, economic, and statistical reasoning can be eliminated through specialized training (Larrick, 2004). All of these cases of specialized training reducing motivated reasoning and other biases can be subsumed under the category of knowledge constraints: certain types of knowledge (e.g., legal rules, economic concepts) constrain us from arriving at conclusions we may otherwise find desirable or in accord with our prior beliefs.
This has yet to be tested in the field of international relations (IR). The present paper aims to answer whether specialized training in IR impedes the accumulation of ideologically partial knowledge, a precursor to, or the cause of, motivated reasoning – potentially adding IR to the knowledge domains in which training has been shown to reduce the effects of bias. To do so, we compare results of bias-sensitive tests of foreign policy knowledge among experts (IR professors) and non-experts, linking these measures with stated foreign policy and IR theory preferences and political opinions. This method does not allow us to test directly for motivated reasoning, but rather for potentially biased stores of knowledge that may lead to motivated reasoning; conversely, an unbiased store of knowledge may provide a “knowledge constraint” impeding motivated reasoning.
Background
One’s store of knowledge can be the target of ego- and belief-protective bias. In a classic paper, Greenwald (1980, p. 604) observed that the “ego fabricates and revises [personal] history, thereby engaging in practices not ordinarily admired in historians,” but commonly employed by totalitarian regimes. These practices – egocentricity, “beneffectance” (the self being held responsible for positive but not negative outcomes), and cognitive conservatism – distort our personal histories, creating a biased store of knowledge. Selective recall can affect cognitions about others, as well as about aspects of “shared reality” (Eitam, Miele, & Higgins, 2013). Shared reality, or social representations (Moscovici, 1961/2008), can include knowledge about international politics or the world system – and the shared nature of such knowledge is what confers upon it greater legitimacy and seeming truthfulness, making it less likely that knowledge tending to contradict such a shared reality or social representation will be remembered. For instance, many Americans share social representations about their country that hold it to be an egalitarian meritocracy; this may be how motivated reasoning prompts over a quarter of the population to believe that the Constitution guarantees a job, and nearly half to believe it guarantees health care and is the source of the communist tenet “from each according to his abilities, to each according to his needs” (Carpini & Keeter, 1996, p. 98).
Knowledge distortions arising from cognitive biases or selective exposure are far from trivial. Knowledge matters, particularly in politics (Friedman, 2005). Differing levels of information profoundly affect collective preferences (Althaus, 1998), voting decisions (Fowler & Margolis, 2014), support for war (Sirin, 2011), and levels of tolerance, political participation, opinion consistency, interest-attitude congruence, and attitude-participation congruence (Carpini & Keeter, 1996, pp. 218-267). Widely shared knowledge, or social representations, concerning a group’s or a country’s history profoundly impact collective identity and how people will react to new political challenges (Liu & Hilton, 2005). Whether one conceives of political ideologies or worldviews as social representations, memeplexes, or schematic structures, they are fundamentally composed of information (plus affect) – as such, the knowledge one has, and the biases and gaps in it, delimits the possible form one’s political ideology can take (Beattie, 2016). Partial knowledge and motivated reasoning are rife in politics and likely play a powerful role in producing political polarization (Shapiro & Bloch-Elkon, 2008).
In a study of Portuguese college students, those who identified more with their national in-group gave greater weight to positive information about their country’s colonial wars while refusing to acknowledge negative information, thereby reducing the collective guilt they experienced (Figueiredo, Valentim, & Doosje, 2011). A study of Canadian Sikhs and Hindus found that those with greater in-group identification (and those experimentally manipulated to identify more strongly with their in-group) remembered fewer incidents of violence and hatred committed by their group than low in-group-identifiers (Sahdra & Ross, 2007). Closer to the present study, Nelson, Adams, and Salter (2013) measured Black and White American college students’ perceptions of racism and knowledge about the history of racism in the U.S. To make measurement of historical knowledge more accurate, the authors adopted a technique from signal detection theory (Stanislaw & Todorov, 1999): providing true and false statements of critical (racist) and celebratory (anti-racist) historical fact, and asking participants to rate them as true or false along with the level of confidence they had in their answers. They found that Black participants had more accurate knowledge (more true hits/correct rejections, fewer false alarms/misses) of the history of racism in the U.S., and that this difference in knowledge mediated Blacks’ more frequent perceptions of systemic racism. In other words, and without addressing causation, accurate knowledge of the history of racism in the U.S. correlated strongly with perceptions of systemic racism today; conversely, a lack of knowledge of the history of racism in the U.S. correlated with an inability to recognize instances of systemic racism today.
The same signal-detection technique was used to investigate whether a lack of knowledge about recent U.S. military and intelligence agency interventions correlated with a preference for a stereotypically Realist (in the international relations theory sense) foreign policy (Snider, 2012). (A “stereotypically Realist” foreign policy is one oriented toward maximizing power, including through the use of force, without restraint from international law or norms; this can also be conceived as “folk realism” [Kertzer & McGraw, 2012]). Counterintuitively, the greater the preference for such a stereotypically Realist foreign policy among the college student sample, the less accurate historical knowledge of recent U.S. military and intelligence agency interventions participants had. This flies in the face of a key pretention of Realist theory, one to which it owes its name: that its greater concern with “historic processes as they actually take place, has earned for the theory … the name of realism” (Morgenthau & Thompson, 1985, p. 4). Instead of being rooted in a greater knowledge of relevant history, participants’ preference for a stereotypically Realist foreign policy correlated with greater ignorance of history, and higher levels of blind (vs. constructive) patriotism. (Blind patriotism is “characterized by unquestioning positive evaluation, staunch allegiance, and intolerance of criticism,” while constructive patriotism is characterized by “questioning and criticism of current group practices that are driven by a desire for positive change” [Schatz, Staub, & Lavine, 1999, p. 153]). These results suggested that rather than emanating from a solid grasp of history and its lessons, preferences for a stereotypically Realist foreign policy may be tied to a certain kind of historical ignorance (which can facilitate motivated reasoning), along with blind attachment to the national in-group. (See, however, Kertzer & McGraw, 2012, in which folk realism was found to correlate with national attachment, but not with a much broader measure of political knowledge).
However, Snider’s (2012) and Kertzer and McGraw’s (2012) studies involved college students, who were unlikely to have significant knowledge about Realism as a body of theory in international relations. Hence, their preferences for a stereotypically Realist foreign policy was unlikely to have reflected any significant knowledge of Realist theory, let alone competing theories in IR, and instead may have been due to a simple affinity for an aggressive, assertive stance in an unfamiliar and dangerous world. Those who spend several years studying for a doctorate degree in IR, and many years as professors of IR, may evince an entirely different relationship between historical knowledge and foreign policy preference. (While not testing for foreign policy knowledge, Rathbun [2012] found that political ideology correlated with paradigm preference in a sample of IR professors). On the other hand, if professional training in IR fails to provide the relevant knowledge constraints, then one might expect those with a blind attachment to the national in-group and a partial knowledge of history to select Realism as their preferred IR theory, and to be less likely to encounter, absorb, or remember historical facts about U.S. foreign policy that would threaten their sense of national identity. This would result in the accuracy of IR professors’ historical knowledge of U.S. foreign policy correlating negatively with their preference for Realism (measured either as self-reported theoretical preference, or as stereotypically realist foreign policy preferences). Or, if the specialized training IR professors receive creates effective knowledge constraints (as legal training seems to do in reasoning about legal problems), then there should be no such negative correlation.
Our first hypothesis, therefore, is H1: while non-experts will perform better at identifying true celebratory and false critical statements and worse at identifying false celebratory and true critical statements, experts will evince no such bias. Given the relative paucity of studies examining different types of experts, we tentatively hypothesized H2: that experts in IR would evince a weaker but significant negative relationship between accuracy of historical knowledge of U.S. foreign policy and preference for Realism. This would indicate that specialized training in IR facilitates the accumulation of a less biased store of knowledge compared to non-experts, but that this reduction in biased knowledge correlates with only a reduction in, not an elimination of, a preference for Realist theory. However, if IR experts do not tend to accumulate ideologically partial knowledge, and the greater knowledge accumulated by IR experts provides a “knowledge constraint” on motivated reasoning, then H3: as levels of blind patriotism and more accurate knowledge of normatively troubling U.S. foreign policy interventions rise among experts, preferences for a hawkish foreign policy will weaken. This would suggest that the reduction in biased knowledge – a gain, relative to non-experts, in “critical” knowledge unflattering to widespread social representations of the national in-group – forms a knowledge constraint impeding the formation of hawkish foreign policy preferences.
Methods
Participants
To collect a sample of professors in the field of international relations, we used a list of academic institutions offering a PhD in political science from Oprisko (2013), supplemented by internet searches for institutions with graduate programs in international relations (or international affairs/politics). We then visited each institution’s web page and collected names and email addresses of all professors with IR as a listed specialty or who teach classes in IR, resulting in a list of 1502 individuals. A total of 400 (27%) clicked through to our online survey from an email request, and 227 individuals (15%) completed the entire survey. (Thus, the final sample of US IR professors should not be considered representative; those who finished the survey may have been more or less knowledgeable than average, or with more or less bias in their knowledge bases). Of these, 10% (22) declined to state their academic title, 31% (69) were at the rank of assistant professor or below, 26% (59) were associate professors, and 34% (77) were full, distinguished, or named professors. 73% were male, 82% were White, and their ages ranged from 28 to 80 (M = 48.03 years, SD = 12.38 years).
On a Left/Liberal to Right/Conservative sliding scale from 0-100, the average self-placement was 30.48 (SD = 20.49); 83% placed themselves to the left of center, and only 12% placed themselves to the right of center. For economic issues, the average was more conservative (M = 37.17, SD = 24.77), and for social issues, the average was more liberal (M = 21.46, SD = 22.86). This is substantially similar to Klein and Stern’s (2006) study of political science professors in the U.S.
To compare this group of experts to non-experts, we collected a separate sample of 258 participants from Mechanical Turk, an appropriate source for a survey of this type (Chandler, Mueller, & Paolacci, 2014). 48% were male, 76% were White, and their ages ranged from 18 to 75 (M = 34.43 years, SD = 13.43 years). On a Left/Liberal to Right/Conservative sliding scale from 0-100, the average self-placement was 40.62 (SD = 28.79). Less than 1% had not completed high school, 19.8% had completed high school, 33.3% had completed some college, 27.1% had completed college, 3.9% had completed some graduate/professional school, and 12.8% had completed a graduate program.
Materials
We devised a total of 28 statements concerning the history of U.S. foreign policy since World War II: half were true (well-documented) and half were false (completely fabricated, not simply true statements with names or dates changed). Among the 14 true and 14 false statements, half were celebratory (“positive”, in keeping with dominant social representations of the U.S. as an upholder of democracy and human rights around the world) and half were critical (“negative”, at variance with dominant social representations of the U.S. role in the world). In total, 7 statements were true/celebratory, 7 were true/critical, 7 were false/celebratory, and 7 were false/critical. The 14 true statements were meant to be fairly representative and the 14 false statements were designed to be plausible, such that measured knowledge of these 28 items could serve as a rough proxy for overall knowledge of recent U.S. foreign policy history.
The foreign policy quiz for the non-experts was a simpler version with 20 questions, with relatively more well-known historical examples (see Supplementary Materials, Suppl. 1, for both surveys). The separate expert version replaced the most widely-known examples with less widely-known examples – this was necessary to test for differences within the group of experts, who could safely be presumed to be more knowledgeable than non-experts – but it made the two quizzes unsuited for direct, one-to-one comparison. Instead, we used the two quizzes to make relative comparisons between the two groups, and absolute comparisons within the two groups.
The critical foreign policy items were “negative” in the sense that they contradict the core American values of human rights and democracy. For instance, most Americans would probably agree that the U.S. was morally superior to the USSR due to the former’s greater respect for human rights. Yet as Coatsworth (2012) observes, during the Cold War the U.S. government directly or indirectly overthrew at least 24 governments in Latin America alone, replacing them with regimes that during the (post-Stalin) period 1960-1990 produced vastly more political prisoners, political executions, and torture victims than the USSR and its East European satellites. This piece of information is precisely the type we would expect to be absent from the knowledge of those strongly identifying with the U.S. national in-group, whether through avoidance of the types of sources likely to contain it, or through motivated reasoning once exposed to it. However, for experts in IR, this knowledge may not be an affront to group identity, and so may be more likely to be absorbed and remembered. That would be so if experts incorporate such knowledge into a wider, exculpatory framework around strategic goals, of the sort Anderson (2015, p. 72) provides in his description of U.S. foreign policy during the Cold War:
Monarchs, police chiefs, generals, sheikhs, gangsters, latifundists: all were better than communists. Democracy was certainly the ideal political system. Where it was firmly established, in the advanced industrial countries, markets were deepest and business was safest. But where it was not, in less developed societies, matters were otherwise. There, if elections were not proof against attempts on private property, they were dispensable. The Free World was compatible with dictatorship: the freedom that defined it was not the liberty of citizens, but of capital – the one common denominator of its rich and poor, independent and colonial, temperate and tropical regions alike.
Hence, “critical” statements were designed to be viewed negatively by those with dominant social representations of the U.S. as a defender of democracy and human rights tout court, but they would not necessarily be viewed similarly by those who believe that democracy and human rights are subordinate to other strategic goals such as power or wealth maximization.
Procedures
First, participants were asked to provide true-false answers and level-of-confidence scores (from 0 – 10) to the foreign policy statements. Afterward, they were asked to provide their level of agreement (from 0 – 10) with a series of stereotypically Realist or hawkish (e.g., “It is acceptable for the U.S. government to engage in covert operations for the sake of American security, even if these actions violate international law”) and stereotypically Liberal or dovish (e.g., “The U.S. should change its policy to more effectively cooperate with the international community and the U.N.”) statements. Next, participants stated their level of agreement (from 0 – 10) with ten prominent theories or paradigms within IR, and were asked with which theory/paradigm they most identified. They then filled out Schatz, Staub, and Lavine’s (1999) blind versus constructive patriotism instrument. Finally, they were asked to rate the relevance to an understanding of U.S. foreign policy (from 0 – 10) of six critical and six celebratory true statements from the foreign policy quiz, followed by demographic questions.
By using measurements from Signal Detection Theory (SDT), we can separate response bias (a general tendency to reply “true” or “false”) from sensitivity (the degree of overlap between the signal (true items) and noise (false items) distributions), for both celebratory and critical items (Stanislaw & Todorov, 1999). One measurement of sensitivity is d’, which describes the distance between the signal (true hits) and noise (false alarms) means in standard deviation units. Positive d’ scores indicate a progressively greater ability to distinguish true from false items, a score of 0 indicates a complete inability to distinguish true from false items, and negative d’ scores indicate response confusion, such as responding “true” when intending to respond “false”. The criterion score, or c, is a means of measuring response bias: also measured in standard deviation units, positive c values indicate a bias in favor of answering “false”, a c value of 0 indicates that neither “true” nor “false” is favored, and negative c values indicate a bias in favor of answering “true”. In other words, d’ can be considered a measurement of attunement to reality (that is, the circumscribed portion of reality covered by the foreign policy quiz), and c can be considered a measurement of response threshold, whether one has a higher or lower threshold for responding true or false. Both d’ and c can be calculated separately for celebratory and critical items, to see whether participants are more accurately attuned to the positive/celebratory or negative/critical sides of US foreign policy history (d’), and whether participants have a higher threshold for answering “true” to celebratory or critical statements on the history of U.S. foreign policy (c).
Using the self-reported level of confidence participants had in each answer, we were able to calculate confidence-adjusted d’ and c scores: correct answers with a confidence of 5 (of 10) or less were counted as misses for true items, and as false alarms for false items. Hence, only when participants were confident in their correct answers for true and false items did they increase the hit rate and reduce the false alarm rate, respectively, for their confidence-adjusted d’ and c scores. Adjusting for confidence reverses some answers, complicating the interpretation of d’ scores – nonetheless, higher values progressively indicate a more accurate “attunement to reality”, even among negative scores.
Results
Research Hypothesis One (H1)
Here we tested whether non-experts, but not experts, performed better at identifying true celebratory and false critical statements, and worse at identifying false celebratory and true critical statements. Among the earlier college student sample, participants had a higher hit rate for true celebratory knowledge than true critical knowledge, and this result was replicated by our MTurk sample of American adults: their hit rate for true celebratory statements was 77%, and for true critical statements was 57%, t(258) = 14.265, p < .001. The professors, on the other hand, displayed no such suggestion of bias in knowledge: a 75% hit rate for true celebratory statements, and a 77% hit rate for true critical statements, t(226) = 1.651, p = .100. Likewise, our non-expert sample had a higher false alarm rate for false celebratory statements (35%) than false critical statements (31%), t(258) = 2.753, p = .006; the professor sample had no statistically significant difference between false alarm rates for celebratory and critical statements (see Figure 1).
Figure 1
Differences in celebratory and critical knowledge.
As a group, the professor sample had a lower c for critical statements than celebratory ones, indicating a lower threshold or greater propensity for answering “true” for statements that contradict dominant, positive social representations about the U.S. upholding democracy and self-determination around the world. Contrariwise, the non-expert MTurk sample had a lower c for celebratory than critical statements, indicating a lower threshold or greater propensity to answer “true” for statements in keeping with dominant, positive social representations about US foreign policy (see Figure 2).
Figure 2
Criterion scores.
In rating the significance of true critical and celebratory statements to an understanding of U.S. foreign policy, while no significant differences emerged among professors as a function of preferred theory, there were important differences between the expert and non-expert samples. Some suggestive evidence for motivated reasoning emerges in both groups, with non-experts, t(241) = 10.38, p < .001, and experts, t(228) = 3.48, p < .001, rating celebratory statements as significantly more relevant than critical statements; however, the gap among experts (6.24 vs. 5.98) was only one quarter the size of the gap among non-experts (7.63 vs. 6.62).
Research Hypothesis Two (H2)
Here we tested whether a preference for Realist theory correlates with biases in relevant knowledge. While Snider’s (2012) earlier college student sample and our MTurk sample were asked about their foreign policy preferences as a measure of “folk realism” or a preference for stereotypically Realist policies, we were able to directly ask the professor sample about their preferred IR theory, allowing us to test for suggestions of biases in knowledge among adherents of different theories.
Due to sample size and the current popularity of various IR theories, only Realism, Liberalism, and Constructivism had sufficient numbers of adherents for statistically significant comparisons of d’ scores (see Supplementary Materials, Suppl. 3 for a frequency table). Self-identified Realists did better than Liberals and Constructivists across the board: a higher d’ overall, adjusted for confidence, and separated into separate d’ scores for critical and celebratory knowledge. Looking at the true hit and false alarm proportions for celebratory and critical knowledge (both true and false) reveals the source of these differences: while there were no significant differences in their respective true hit rates for (true) critical and celebratory statements, Realists outperformed Liberals and Constructivists at correctly rejecting (false) critical and celebratory statements (see Table 1) .
Table 1
Differences in d’ (or Accurate Attunement to U.S. Foreign Policy History)
| Index, Statment Type | Unadjusted
|
Confidence-Adjusted
|
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Realism (50)
|
Liberalism (78)
|
Constructivism (42)
|
Realism (50)
|
Liberalism (78)
|
Constructivism (42)
|
|||||||
| M | SD | M | SD | M | SD | M | SD | M | SD | M | SD | |
| d’ | ||||||||||||
| Overall | 1.27** | 0.53 | 0.96** | 0.54 | 0.95** | 0.55 | -0.13** | 0.79 | -0.57** | 0.92 | -0.70** | 1.07 |
| Celebratory | 1.27** | 0.70 | 0.96* | 0.63 | 0.97† | 0.75 | -0.50† | 0.90 | -0.83† | .1.1 | -0.90† | 1.20 |
| Critical | 1.42* | 0.73 | 1.03** | 0.78 | 1.03* | 0.78 | 0.26*** | 0.99 | -0.40*** | 1.10 | -0.54*** | 1.20 |
| Hit Rate | ||||||||||||
| True Celebr. | 0.74 | 0.18 | 0.73 | 0.15 | 0.73 | 0.16 | 0.45 | 0.22 | 0.43 | 0.23 | 0.41 | 0.26 |
| True Critical | 0.77 | 0.13 | 0.74 | 0.17 | 0.75 | 0.14 | 0.62† | 0.19 | 0.55† | 0.21 | 0.54 | 0.24 |
| False Alarm | ||||||||||||
| False Celeb. | 0.32* | 0.19 | 0.41* | 0.20 | 0.41* | 0.21 | 0.63 | 0.21 | 0.69 | 0.22 | 0.69 | 0.25 |
| False Critical | 0.29* | 0.21 | 0.39* | 0.22 | 0.41* | 0.22 | 0.53* | 0.26 | 0.67* | 0.25 | 0.69** | 0.25 |
†p ≤ .10. *p ≤ 05. **p ≤ .01. ***p ≤ .001, two-tailed, Liberalism/Constructivism compared to Realism, Realism compared to Liberalism.
Correlations between level of agreement with IR theories and d’ (and confidence-adjusted d’) show few significant relationships: only agreement with Realism and the English School show significant (but weak) correlations with d’ scores. But as a predictor of agreement with the various IR theories along with hawkish-to-dovish policy orientation, political identification, and blind and constructive patriotism, d’ marginally-significantly predicts agreement only with the English School, β = 0.125, t(226) = 1.824, p = .070, and Marxism, β = 0.105, t(226) = 1.650, p = .100, and confidence-adjusted d’ significantly predicts agreement only with the English school, β = 0.182, t(226) = 2.683, p = .008. Endorsement of different IR theories thus does not predict level of accuracy in historical knowledge of US foreign policy, excepting the English School (and to a lesser degree, Marxism) (see Supplementary Materials, Suppl. 3 for all results).
Criterion (c) scores show fewer significant differences between adherents of the three most popular theoretical perspectives in our sample. Whether confidence-adjusted or not, c for Realists, Liberals, and Constructivists is negative overall, confidence-adjusted, and for both celebratory and critical statements, indicating a higher threshold for answering “false”. Compared to Realists, Liberals and Constructivists had a lower c across the board, including confidence-adjusted c, and for both critical and celebratory statements, indicating a slightly higher propensity to answer “true”. This does not suggest differential scrutiny in the service of motivated reasoning.
At the more granular level of confidence-adjusted individual questions, some interesting differences emerge. While no significant differences emerged among the true celebratory statements, among the true critical statements, Realists did significantly better (98%) than Liberals and Constructivists (83% for both) at confidently acknowledging the US role in overthrowing Iran’s former Prime Minister Mossadegh, and at confidently acknowledging the U.S. role in Indonesia’s massacre of 500,000 to one million alleged communists (while only 40% of Realists and 24% of Liberals confidently answered “true”, five of seven Marxists in the sample did so). More Realists than Liberals and Constructivists confidently (and correctly) denied that the U.S. imposed harsh economic sanctions on Iraq in response to Saddam Hussein’s use of poison gas against the Kurdish town of Halabja in 1988 (54%, 35%, and 31%, respectively); that the United States used defoliant-spraying planes in 1967 to destroy Bangladeshi crops in an attempt to destabilize the communist-friendly government (68%, 36%, and 31%, respectively); and that the U.S. Air Force provided assistance to French troops in Indochina in 1951 by bombing cities considered to be strongholds of the communist resistance (70%, 42%, and 43%, respectively).
Research Hypothesis Three (H3)
Here we tested the relationship between blind patriotism, knowledge, and foreign policy preferences, looking for indications that an unbiased knowledge base, even in the presence of relatively high levels of blind patriotism, may impede the formation of hawkish foreign policy preferences. In the aforementioned college student sample, blind patriotism was positively correlated with a measure of stereotypically Realist (hawkish) foreign policy preferences. This result was replicated by both our MTurk and professor samples: blind patriotism was also correlated with hawkish policy preferences for non-experts (r = .566, p < .001) and experts (r = .513, p < .001). However, while support for a hawkish foreign policy moderately correlates with professors’ self-reported agreement with Realism as a body of theory (r = .399, p < .001), blind patriotism does not (r = .041, p = .539). In fact, only one of the ten IR theories we asked professors to rate their agreement with had any significant correlation whatsoever with blind patriotism: Marxism, which was negatively, weakly related (r = -.167, p = .009) (see Supplementary Materials, Suppl. 3). Controlling for political identification in separate regression analyses, only increased agreement with Realism predicts more hawkish policy preferences; increased agreement with Critical Theory, Postcolonialism, Poststructuralism (all three statistically-significantly), Green Theory, and Feminism (both at marginal levels of significance) predict more dovish policy preferences; and level of agreement with Liberalism, Constructivism, the English School, and Marxism have no significant predictive value (in all regressions, self-placement further to the Right predicts more hawkish preferences; see Supplementary Materials, Suppl. 2).
A regression analysis of the earlier college student sample found that among predictors of support for a hawkish foreign policy, only blind patriotism and d’ for celebratory knowledge reached statistical significance (other predictors included constructive patriotism, d’ for critical knowledge, political identification, and interaction terms for d’ celebratory and d’ critical by blind patriotism). The same regression analysis performed on our MTurk sample found support for a hawkish foreign policy was predicted by rightwing political identification, blind patriotism, celebratory d’, and at marginal significance, a blind patriotism by critical d’ interaction term. Our professor sample differed in interesting ways: like the non-experts, support for a hawkish foreign policy was predicted by rightwing political identification and blind patriotism, but also by an interaction of blind patriotism and critical d’ that differed in direction across the two samples (see Table 2).
Table 2
Predictors of Dovish-to-Hawkish Policy Preference
| Independent Variable | Non-Expert MTurk Samplea |
Expert Professor Sampleb |
||||||
|---|---|---|---|---|---|---|---|---|
| B | SE | β | t | B | SE | β | t | |
| Political ID (Left 0, Right 100) | 0.017** | 0.005 | 0.196** | 3.160 | 0.051*** | 0.004 | 0.600*** | 12.049 |
| d’ Celebratory | 0.381** | 0.131 | 0.165** | 2.908 | -0.066 | 0.137 | -0.030 | -0.485 |
| d’ Critical | 0.271 | 0.138 | 0.093 | 1.569 | 0.225† | 0.136 | 0.103† | 1.658 |
| Blind Patriotism | 1.126*** | 0.162 | 0.436*** | 6.954 | 0.521*** | 0.105 | 0.257*** | 4.979 |
| Blind Patriotism x d’ Celebratory | -0.046 | 0.178 | -0.014 | -0.258 | 0.216 | 0.161 | 0.102 | 1.346 |
| Blind Patriotism x d’ Critical | 0.334† | 0.192 | 0.102† | 1.737 | -0.379* | 0.173 | -0.164* | -2.197 |
Note. Negative values indicate dovish orientation; positive values indicate hawkish orientation. B = unstandardized coefficient; β = standardized coefficient.
aN = 258, Adj. R2 = .323, F = 20.03***. bN = 226, Adj. R2 = .546, F = 46.36***.
†p ≤ .10. *p ≤ 05. **p ≤ .01. ***p ≤ .001, two-tailed. All variables except Political ID were centered.
This suggests a fundamentally different relationship between knowledge and hawkish foreign policy preference among experts and non-experts: while accuracy of celebratory knowledge predicts a hawkish orientation for non-experts, it does not for experts. Additionally, the interaction term for blind patriotism and critical d’ is negative for experts and positive for non-experts, suggesting that this particular type of knowledge may provide a constraint on motivated reasoning among experts. That is, among experts, as blind patriotism increases – holding knowledge of “critical” aspects of US foreign policy history constant – so too does support for a hawkish foreign policy. However, as blind patriotism increases along with accurate knowledge of critical foreign policy history, support for a hawkish foreign policy is reduced among experts (see Figure 3)
Figure 3
Comparing non-experts and experts on relationships between blind patriotism, knowledge, and policy preferences.
Discussion
The pattern that emerges from a comparison between our expert and non-expert samples is that non-experts evince clear signs of biased knowledge, while IR experts as a group do not. Non-experts were better at recognizing true statements that reflect positively versus negatively on their national in-group, while experts were slightly better at recognizing the true critical statements. Likewise, non-experts were more likely to mistake false celebratory statements as true compared to false critical statements; experts evinced no significant differences among the celebratory and critical false statements. This pattern is reflected in the c scores of the two groups; H1 is supported by these results.
In Snider’s (2012) analysis of college students, hawkish foreign policy preferences were used as a proxy for agreement with Realism as a theory, and were found to correlate with blind patriotism – suggesting that one’s opinions on foreign policy are not the result of dispassionate analysis of reality but are at least partially due to one’s emotional attachments to a national in-group. Likewise, in both our expert and non-expert samples, blind patriotism correlated with hawkish (or stereotypically Realist) foreign policy preferences. However, among the professors, blind patriotism did not correlate with their level of agreement with Realism – or any theory (except for a negative correlation with Marxism). Self-identified Realists, instead of displaying greater bias in knowledge than adherents of other theories, actually did better than adherents of all other theories besides the English School. This suggests that those with years of training in IR come to adopt a theoretical perspective due to factors other than national in-group bias. H2 receives no support from these results.
Hawkish foreign policy preferences were predicted by blind patriotism and political conservatism for both experts and non-experts. However, among non-experts, greater attunement to positive aspects of U.S. foreign policy history (celebratory d’) predicted hawkish policy preferences, while for experts, critical d’ is a (nearly significant) predictor. Furthermore, among non-experts the higher the level of blind patriotism and the more accurate knowledge of violent, anti-democratic U.S. interventions in the world, the higher is the level of support for a hawkish foreign policy (approaching the conventional level of significance). But among experts, as levels of blind patriotism and accurate knowledge of “negative” foreign policy interventions increase, support for a hawkish foreign policy decreases. This suggests that for experts, knowing more about examples of violent and anti-democratic instances of U.S. foreign policy provides a “knowledge constraint” on the motivated reasoning blind patriotism might otherwise spur. H3 receives support from these results.
Why might greater knowledge of “negative” foreign policy history affect experts and non-experts differently? First of all, our test for experts comprised less widely known historical facts than our test for non-experts; the two tests were meant to measure differences in knowledge within rather than between expert and non-expert groups, and to do so the test for experts needed to be harder. Hence, if our tests succeeded at being fairly representative samples of a much larger body of historical knowledge, doing well on the expert test would suggest more extensive knowledge of foreign policy – and a greater knowledge constraint on motivated reasoning – than the non-expert test. Nonetheless, non-experts with higher levels of blind patriotism and who knew more about negative foreign policy history relative to other non-experts had more hawkish policy preferences; whereas experts with higher levels of blind patriotism and who knew more about negative foreign policy history than other experts tended to have less hawkish policy preferences.
Within the group of IR experts, there were interesting differences in knowledge between adherents of different theories. Morgenthau’s characterization of realism as a theory set apart by its greater attunement to historical reality receives some support from our results: self-identified Realists performed better than their Liberal and Constructivist counterparts (higher d’, c closer to 0). Liberals and Constructivists were just as good as Realists at identifying true critical and celebratory facts, but did worse at identifying false statements – both critical and celebratory. This would suggest not motivated reasoning, but relatively less familiarity with U.S. foreign policy history.
However, when adjusted for confidence, a relevant change emerges: Realists’ better ability to separate celebratory fiction from fact disappears, and Liberals become less likely to correctly identify critical facts as true. (One of these facts, U.S. support for the massacre of 500,000 to one million alleged Communists in Indonesia, was rather infamously [e.g., Holmes, 2002] absent from Samantha Power’s Pulitzer Prize-winning book on mass atrocities and U.S. foreign policy). Part of the confidence-adjusted effects are due to the sample’s Realists having a higher average age than Liberals and Constructivists, and confidence correlating positively with age; but in separate ANCOVA analyses age was never the sole significant predictor. Differences of this sort – like Marxists doing significantly better than average at correctly identifying true critical and false celebratory statements, but worse than average at correctly identifying false critical and true celebratory statements – do suggest a degree of biased knowledge even among experts.
These results concern biases in knowledge, which is a precursor to motivated reasoning; they do not, however, provide evidence of motivated reasoning itself. Whereas motivated reasoning occurs on the fly, bias in knowledge (if it is not merely selective recall) is the result of a long period of accumulating information wherein some information is favored over other information. Biases in knowledge can also be the result of selective exposure: the choice to read a book known to contain information threatening to a positive view of the national in-group, versus a book whose content is known to affirm such a positive view. Additionally, various facets of the sociology of IR academia may also be involved: different theories being in vogue at different times, and different episodes of U.S. foreign policy receiving attention in the news media and scholarship during the formative years of various cohorts of IR scholars. Hence it is impossible to be certain about whether any bias in knowledge derives from motivated reasoning, selective recall, selective exposure, or effects of the sociology of the profession.
It is intuitively reasonable that those attracted to Marxism would choose to expose themselves to information that may put the national in-group in a bad light – or, that those who have been exposed to such information would choose Marxism as their favored IR theory. With Realism and Liberalism, however, no such intuitive connection springs to mind. College students first introduced to the theories may well associate Realism with a rightwing concern for maximizing national power particularly through military means, and Liberalism with a leftwing concern for ameliorating international conflict through institutions and organizations. But by accumulating more information about IR, one becomes exposed to a wide spectrum of political orientations within each theory: for instance, the arch-leftist Noam Chomsky adopts a characteristically Realist perspective in his political analyses (Osborn, 2009), and the arch-Realist E.H Carr adopted a characteristically Marxist perspective in his critique of the ‘harmony of interests’ (Carr, 1964).
Furthermore, whether biases in knowledge arise primarily from selective exposure or selective remembering, it is less likely to affect the political commitments of IR experts as it would non-experts. For a college student, it may cause uncomfortable cognitive dissonance to learn that the U.S. government supported Suharto’s massacre and the overthrow of Mossadegh, Arbenz, and Allende, bombed dams in Korea to starve the population, illegally mined Nicaraguan harbors, and had Nelson Mandela on its list of terrorists for nearly 20 years. This information may contradict the student’s understanding of the U.S. as a force for peace and democracy in the world, and either provoke a change in such an understanding (and a move toward a more dovish orientation) or a questioning of the veracity of such information (and a rejection of true, critical information). A student being trained in IR, however, will learn interpretive frameworks – IR theories – which can remove this cognitive dissonance completely, and allow such information to be incorporated without a drastic reorganization of schematic structures. So while a non-expert may find such critical information difficult to incorporate into dominant, celebratory social representations about U.S. foreign policy, experts can do so easily: critical information can be contained under the category of “unpleasant but necessary” (“necessary” given certain assumptions about the nature of the world system), leaving a rosy view of the national in-group unaffected.
Realism in particular would seem to provide an ideal framework: what atrocity cannot be justified by recourse to the anarchic nature of the world system and each state’s vital need to maximize power or flirt with extinction? Liberalism, while perhaps less obviously ideal for such a task, certainly provides a framework of its own to justify violent military intervention (Desch, 2008). A possible comparison between Realists and Liberals is echoed in General Curtis LeMay’s comments on the Korean War:
We slipped a note kind of under the door into the Pentagon and said, “Look, let us go up there … and burn down five of the biggest towns in North Korea – and they're not very big – and that ought to stop it." Well, the answer to that was four or five screams – “You'll kill a lot of non-combatants, and it’s too horrible.” Yet over a period of three years or so … we burned down every [sic] town in North Korea and South Korea, too.… Now, over a period of three years this is palatable, but to kill a few people to stop this from happening – a lot of people can't stomach it. (quoted in Cumings, 1997, p. 298)
The stereotypically Realist perspective would seem to align with LeMay (secure a national interest using the most effective means), and the stereotypically Liberal perspective would seem to align with the Pentagon’s reaction (refuse means that offend moral norms and international law) – but the real-world result was the same. As Pery Anderson (2015, p. 195) puts it: “Liberal internationalism is the obligatory idiom of American imperial power. Realism, in risking a closer correspondence to its practice, remains facultative and subordinate.” Of course, Realists as well as Liberals can take anti-war stands (e.g., Page & Bouton, 2006, pp. 6-7) – both IR theories offer an overarching framework within which foreign policies of all shades of morality (and standards by which to judge morality) can be comfortably incorporated. Non-experts, without the accommodating schematic structures provided by a deep knowledge of a Realism or Liberalism, evidently have greater trouble incorporating knowledge about U.S. foreign policy history that contradicts prevalent, rosy social representations of it. This may be why more evidence of biased knowledge was found among the non-expert sample compared to the expert sample.
Conclusion
We are at pains to emphasize that our results do not speak to the value or “scientificity” of any IR theory. We agree with Jackson (2011, p. 3) that within IR debates, the term “science” has been used “in extremely cavalier ways, standing-in generally as the positive pole of a contrast that an author wishes to draw between her or his approach to generating and evaluating claims about world politics and some reviled alternative.” This is unfortunate, since not only have contemporary philosophers of science abandonedi the dominant approach to scientific inquiry within IR, neopositivism, but the entire enterprise of the philosophy of science consists in explaining obviously successful knowledge production in terms of its scientific character – and “[n]o such obvious successes exist in IR, which changes the terms of the debate quite radically” (Jackson, 2011, p. 16). Therefore, we echo Jackson’s call for paradigmatic and methodological pluralism within IR, in the hope that a diversity of ways of producing knowledge about the world will prove most useful.
If anything, our bias lies in preferring a mix of IR theories, borrowing insights and methods from each as individual research questions may require. From this perspective, Realism offers the insight that power is likely to be the most influential factor in the world system; Liberalism that economic relationships, institutions and international organizations are powerful influences in their own right; Constructivism that ideas too are consequential; the English School and Postcolonialism that an understanding of history is essential to any explanation of the international system; Marxism that differing levels of economic power both within and between countries affect global politics; Critical Theory and Poststructuralism that the assumptions of dominant theories must always be interrogated; Feminism that gender powerfully affects both international politics and its study; and Green Theory that on a finite planet, ecological matters are of primary importance to international politics. This may place us squarely within the scientific realist camp, particularly in Bhaskar’s (1975/2008, p. 18) formulation: in complex, open systems like that of international relations, statements of laws are unlikely to obtain, and are more accurately conceived as statements of tendencies, which “may be possessed unexercised, exercised unrealized, and realized unperceived (or undetected) by men; they may also be transformed.” As such, only a diversity of methodologies and theoretical constructs can hope to accurately describe and generate knowledge about the open, complex system of international politics.
Although our results and their interpretation suggest possible biasing factors in IR theories, like ideology and biases in knowledge, these do not make such theories any less “scientific”. Solomon (2001) points out that calling such factors “biasing” is inappropriately judgmental in the first place, arguing that since “motives, values, ideologies and so forth can be conducive to scientific success, they deserve, no less than traditional values of science such as simplicity, fruitfulness, consistency, etc., the status of ‘scientifically rational’” (p. 53). In her terms, such factors are better described by the epistemically neutral “decision vectors”, all of which may or may not prove conducive to scientific success, as the history of science demonstrates.
On the one hand, evidence that IR experts may be less susceptible to motivated reasoning (through biases in knowledge) than the broader population is a normatively pleasing result. However, if the most troubling aspect of motivated reasoning is that an accumulation of partial knowledge affects judgments that rely on one’s stock of knowledge, producing distortions in reasoning that look as if they were motivated by non-rational emotional commitments, then this result may not be quite so comforting. If one’s knowledge of anti-democratic, violent interventions into other countries’ affairs does not influence one to avoid such interventions because said knowledge is incorporated into a broader, exculpatory framework – then an absence of biases in knowledge may be effectively irrelevant. Michael Mandelbaum (2002, p. 67) explains:
Whereas for the foreign-policy elite, the need for American leadership in the world is a matter of settled conviction, in the general public the commitment to global leadership is weaker…. That commitment depends on a view of its effects on the rest of the world and the likely consequences of its absence. These are views for which most Americans … lack the relevant information…
If the information considered “relevant” comprises an exculpatory framework of unduly optimistic/pessimistic or self-serving assumptions about U.S. global leadership’s “effects on the rest of the world and the likely consequences of its absence,” then the distinction between partial, biased knowledge on the one hand, and unbiased knowledge encased within a justificatory schematic structure on the other, is without much of a difference.
Lacking the information Mandelbaum considers relevant, the U.S. public desires a global order whose leadership is shared among the community of nations (Page & Bouton, 2006) – not one whose leadership is dominated by its own government, and influenced primarily by business leaders (Jacobs & Page, 2005) with the goal of maximizing the wealth and power of the U.S. economic elite (Shoup, 2015). Perhaps the biases in knowledge evident among members of the public is not as normatively troubling as it may seem, if the elimination of such biases in knowledge among experts comes at the price of incorporating previously threatening information into an exculpatory framework founded on assumptions so convenient to the in-group’s interests as to be of dubious validity (e.g., Trivers, 2011).
However, our results suggest that this does not always happen among IR experts: instead, among those with higher levels of blind patriotism, greater knowledge of “negative” foreign policy interventions seems to produce a knowledge constraint preventing reasoning from arriving at the hawkish policy preferences that would justify these sorts of interventions. This suggests that knowledge itself – so long as it is not biased in a celebratory direction – may produce a knowledge constraint on motivated reasoning among experts in IR. While we did find some evidence of political ideology influencing support for various IR theories (or, possibly, support for and exposure to various IR theories influencing political ideology), among experts this does not seem to be a factor of a biased store of knowledge.