A growing body of evidence has implicated conspiracist ideation in the rejection of scientific propositions. Internet blogs in particular have become the staging ground for conspiracy theories that challenge the link between HIV and AIDS, the benefits of vaccinations, or the reality of climate change. A recent study involving visitors to climate blogs found that conspiracist ideation was associated with the rejection of climate science and other scientific propositions such as the link between lung cancer and smoking, and between HIV and AIDS. That article stimulated considerable discursive activity in the climate blogosphere—i.e., the numerous blogs dedicated to climate “skepticism”—that was critical of the study. The blogosphere discourse was ideally suited for analysis because its focus was clearly circumscribed, it had a well-defined onset, and it largely discontinued after several months. We identify and classify the hypotheses that questioned the validity of the paper’s conclusions using well-established criteria for conspiracist ideation. In two behavioral studies involving naive participants we show that those criteria and classifications were reconstructed in a blind test. Our findings extend a growing body of literature that has examined the important, but not always constructive, role of the blogosphere in public and scientific discourse.
U.S. Senator James Inhofe, July 28, 2003
(149 Congressional Record S10012 — Science of Climate Change, p. 11)
The discovery by John Tyndall that CO2 is a greenhouse gas dates back over 150 years, and it was recognized more than a century ago that industrial CO2 emissions may alter the planet’s climate. During the last two decades, the scientific evidence for the fact that humans are interfering with the climate has become unequivocal. The vast majority of domain experts agree that the climate is changing due to human CO2 emissions (
Given this broad agreement on the fundamentals of climate science, what cognitive mechanisms underlie the dissent from the consensus by a vocal minority of people (
A second variable, which we focus on here, is conspiracist ideation; that is, a person’s propensity to explain a significant political or social event as a secret plot by powerful individuals or organizations (
In the case of climate change, several qualitative analyses have shown that denial is suffused with conspiratorial themes, for example when dissenters are celebrated as “Galileos” who oppose a corrupt scientific “establishment”. Researchers have suggested that the dissenters’ creation of such a shared social reality provides a stage upon which public climate denial can unfold (
At a behavioral level,
Author | Title | Citation |
---|---|---|
James Inhofe | ||
Larry Bell | ||
Andrew Montford | ||
Larry Solomon | ||
Pat Michaels and Robert Balling | ||
Brian Sussman | ||
Rael Jean Isaac |
The first study sampled visitors to climate blogs (
Although the results of LOG12 were not unexpected on the basis of prior research, the paper caused considerable discursive activity on “skeptical” Internet blogs. Some climate blogs arguably contain conspiracist themes, for example when expressing the belief that “. . . the alarmists who oversee the collection and reporting of the data simply erase the actual readings [of temperatures] and substitute their own desired readings in their place” (
To provide full personal and historical context, we briefly summarize the history of the present article. Two of the present authors (S.L. and K.O.) also authored LOG12 and LGO13, and they therefore became aware of the initial “skeptical” blog response to LOG12. Because this public response seemed of sufficient relevance to the scholarly questions surrounding the involvement of conspiracist discourse in the rejection of science, a research project commenced that analyzed this public discourse. The relevance of this project was further underscored by repeated non-public attempts to prevent or delay the publication of LOG12 while it was in press and even after its publication. For example, the editor of the journal that published LOG12 was approached with an ethical complaint, and the request for retraction of LOG12 while it was in press, when the ethical approval of the study became public through a freedom-of-information request. An early summary of those events was reported elsewhere (
The analysis of the blog-centered public response to LOG12 was published as a thematic analysis in an online open access journal (
The withdrawal of Recursive Fury engendered a number of further events, including the very public resignation of three editors of the journal
The present article reports an anonymized and updated version of the thematic analysis reported by
The purpose of the first study was to examine public discourse on climate “skeptic” blogs in response to the publication of LOG12 and to classify the content of this discourse into distinct themes that might potentially be considered conspiracist. For brevity, we use the term “blogosphere” from here on to characterize the collective activity on climate “skeptic” blogs. Our approach follows common narrative methods and is best classified as a “thematic analysis”. In a thematic analysis, the focus is exclusively on content—as opposed to the construction of a single “story” or analysis of linguistic structure. Emphasis in thematic analysis is on “(...) ‘what’ is said, rather than ‘how’, ‘to whom’, or ‘for what purpose’” (
Internet activity related to LOG12 was sampled using Google search. Results were limited to English-speaking sites and text. Comparative media analyses have shown that climate “skepticism” is particularly prevalent in Anglophone countries (
If new blog posts were discovered that featured links to other relevant blog posts not yet recorded, these were also included in the analysis. To ensure that the collection of material was as exhaustive as possible, Google was searched for links to the originating blog posts (i.e., first instances of a claim), thereby detecting any further references to the original content or deviations from it.
Although the search encompassed the entire (English-speaking) web, it became apparent early on that the response of the blogosphere was focused around a number of principal sites. To formally identify those sites, we began by analyzing the 30 most-frequently read “skeptic” websites, as identified by Alexa rankings. Alexa is a private company, owned by
Website | Google hitsa | Blog Postsb |
---|---|---|
A | 747 | 11 |
B | 82 | 8 |
Cd | 40 | 3 |
D* | 36 | 11 |
E | 33 | 4 |
Fc | 30 | 7 |
G*d | 20 | 17 |
H | 18 | 6 |
I | 16 | 0 |
J | 13 | 2 |
aTotal number of hits on each site to “S.L.” that fell within the period 28 August-18 October 2012.
bTotal number of blog posts featuring recursive theories about LOG12 posted within the period 28 August-18 October 2012.
cThis blog is not among the top-30 “skeptic” sites but was a principal player in the response to LOG12 because its proprietor launched several freedom-of-information requests.
dThese blogs reposted content from other blogs but published no original content of their own.
Blog posts or comments that mentioned recursive theories—without necessarily endorsing them—were excerpted, with each excerpt representing a mention of the recursive theory (see
ID | Na | Date | Sourceb | Summary of hypothesis | Criteriac |
---|---|---|---|---|---|
1 | 27 | 29 Aug | A | Survey responses “scammed” by warmists | QM, PV, MbW, SS |
2 | 13 | 29 Aug | A | “Skeptic” blogs not contacted | QM, OS, PV |
3 | 3 | 3 Sep | B | Presentation of intermediate data | QM, OS, MbW, UCT |
4 | 3 | 4 Sep | C | “Skeptic” blogs contacted after delay | QM, OS, MbW, NoA, UCT |
5 | 5 | 5 Sep | D | Different versions of the survey | QM, MbW, UCT |
6 | 3 | 6 Sep | D | Control data suppressed | QM, NoA |
7 | 3 | 10 Sep | D | Duplicate responses from same IP number retained | OS, MbW |
8 | 2 | 14 Sep | D | Blocking access to authors’ Websites | QM, PV, NoA |
9 | Various | Various | Miscellaneous hypotheses | See text | |
10 | 3 | 12 Sep | E | Global activism and government censorship | QM, PV, SS |
aTotal number of mentions in corpus.
bAttribution is based on where and by whom a hypothesis was first proposed in public. Note that the initial proposal of a hypothesis need not imply conspiratorial content: Hypotheses are listed only if the collective response of the blogosphere over time assumed conspiracist attributes.
cQM = Questionable Motives; OS = Overriding Suspicion; PV = Persecuted Victim; MbW = Must be Wrong; NoA = No Accident; SS = Self Sealing; UCT = Unreflexive Counterfactual Thinking.
During the search, a classification scheme of recursive hypotheses was developed that tentatively assigned each comment in the corpus to a hypothesis. The classification of hypotheses necessarily evolved as the search proceeded and as new hypotheses were discovered.
To process the corpus and to test for the presence of conspiracist discursive elements, we derived six criteria from the existing literature (see
The first criterion is that the presumed motivations behind any assumed conspiracy are invariably nefarious or at least questionable (
A corollary of the first criterion is that the person engaging in conspiracist discourse perceives and presents her- or himself as the victim of organized persecution. At least tacitly, people who hold conspiratorial views also perceive themselves as brave antagonists of the nefarious intentions of the conspiracy; that is, they are victims but also potential heros. The theme of the victimization and potential heroism features prominently in science denial, for example when isolated scientists who oppose the scientific consensus that HIV causes AIDS are presented as persecuted heros and are likened to Galileo (
Third, conspiracist ideation is characterized by “(...) an almost nihilistic degree of skepticism” (
Fourth, the overriding suspicion is often associated with the belief that nothing happens by accident (e.g.,
Fifth, the underlying suspicion and lack of trust contribute to a cognitive pattern whereby specific hypotheses may be abandoned when they become unsustainable, but those corrections do not impinge on the overall abstraction that “something must be wrong” and the “official” account must be based on deception (
Finally, contrary evidence is often interpreted as evidence
Concerning the rejection of climate science, a case in point is the response to events surrounding the illegal hacking of personal emails of climate scientists, mainly at the University of East Anglia, in 2009. Selected content of those emails was used to support the theory that climate scientists conspired to conceal evidence against climate change or manipulated the data (see, e.g.,
The hypotheses that evolved during the thematic analysis are classified into distinct clusters in
The unit of analysis for this study was the hypothesis, not the individual comment. Hypotheses introduced by one individual were typically picked up by others. Hence, all entries in
Whenever people express their opinions it cannot be ruled out that they are “faking” their responses by providing answers that are intended to please (or deceive) the experimenter. This possibility may be exacerbated with Internet surveys. In a politically charged context, such as climate change, the further risk arises that some respondents may “scam” the survey by “faking” responses to deliver a “desired” outcome.
This risk was rapidly perceived by commenters in the blogosphere, and almost immediately (on 29 August 2012) the concern was expressed that the LOG12 survey was (a) designed to link “skeptics” with “conspiracy nutters” and that therefore (b) some “alarmist” respondents might dutifully perform as expected. [DC 3].
The notion of “scamming” took center-stage in the blogosphere’s response to LOG12. On numerous blogs, it appeared to be taken for granted that the data were “faked” or “scammed.” In one blog post that repeated the words “scam” or “scammed” 21 times (the post ran to approximately 5,100 words), the author asserted that “almost certainly” some respondents of the survey were caricaturing climate “skeptics.” [DC 79].
During exploration of this hypothesis, initial focus in the blogosphere rested on responses to the LOG12 survey items that targeted conspiracist ideation, with the assertion that the few people who endorsed all (or all but one) conspiracy theories (
This assertion transmuted into several additional scamming hypotheses: On 8 September, a second type of purportedly fake responses was reported involving the participants (
On 23 September, it was reported that a further 48 participants had been identified who registered zealous support for free market ideology. This zealous support was taken to imply that those responses, too, represented “scammed” data as the overall incidence of extreme support for the free market in the LOG12 data was greater than in an alternative survey conducted on a “skeptic” blog after the controversy over LOG12 erupted [DC 78]. It was concluded that the data from these apparent zealots were caricatured responses of “skeptics” that were actually provided by people who endorsed climate science [DC 159].
The pursuit of the scamming hypothesis without clear
Initial attention of the blogosphere also focused on the method reported by LOG12, which stated: “Links were posted on 8 blogs (with a pro-science stance but with a diverse audience); a further 5 ‘skeptic’ (or ‘skeptic’-leaning) blogs were approached but none posted the link.” Speculation immediately focused on the identity of the 5 “skeptic” bloggers. Within short order, 25 “skeptical” bloggers had come publicly forward [DC 2] to state that they had not been approached by the researchers. Of those 25 public declarations, five were by individuals who were invited to post links to the study by the LOG12 team in 2010. Two of these bloggers had engaged in correspondence with the research assistant for further clarification.
This apparent failure to locate the “skeptic” bloggers led to allegations in blog posts and comments of research misconduct by the LOG12 team. Those suspicions were sometimes asserted with considerable confidence, such as when a commenter asserted it was “known” that contacting the blogs was “made up.” [DC 21]. One blog comment airing the suspicion that “skeptic” bloggers had not been contacted also provided the email address to which allegations of research misconduct could be directed at the host institution of S.L. (Several such allegations ensued.) This comment was posted by an individual who had been contacted twice by the research team.
The names of the “skeptic” bloggers became publicly available on 10 September 2012 (by which time one individual had already identified himself in public), on a blog post by S.L.;
First, the hypothesis that bloggers were not contacted was abandoned gradually. For example, one blogger opined that even if the blogs had been contacted, S.L. must have known that they would refuse to post the link because the results would have been distorted to try to “harm skeptics.” [DC 99].iii This statement explicitly imputes a pervasive stance of suspicion among “skeptic” bloggers (criterion Overriding Suspicion) because the statement presumes that any blogger would assume that the survey intended to “harm skeptics.” This statement also illustrates the self-perception as a victim of persecution (Persecution-Victimization).
Similarly, it was pointed out that a research assistant, rather than one of the LOG12 authors, had emailed “skeptic” bloggers, whereas S.L. was named in emails to pro-science blogs. [DC 103]. This “inconsistent delivery” sub-hypothesis lasted for 48 hours (11–13 September) and meets criteria Must be Wrong, Nothing is an Accident, and Questionable Motives. (The assistant’s involvement had explicit ethics approval.)
Notwithstanding the abandoning of the initial “no-contact” hypothesis, the allegation that the survey was
It is notable that concerns about the representativeness of the LOG12 sample were rarely mentioned outside the context of the hypotheses just reviewed. Only two blog comments noted that because “skeptic” blogs did not post links to the survey, the LOG12 sample may have been skewed towards people who endorse the science, without also accompanying that critique with a hypothesis of nefarious intentions or malfeasance on the part of LOG12.
Once hypothesis-shifting was complete, several new hypotheses emerged in short order to counter the conclusions of LOG12. Several of those hypotheses were based on what we call unreflexive counterfactual thinking; that is, the hypothesis was built on a non-existent, counterfactual state of the world, even though knowledge about the true state of the world was demonstrably available at the time.
S.L. presented a talk at Monash University in Melbourne on 23 September 2010. The slides for that talk were posted on the web on 27 September 2010 and contain a single brief reference (10 words: “conspiracy factor without climate item predicts rejection of climate science”) to the LOG12 data, based on the responses received by that date (nearly the entire sample).
Because this date fell within three days of the second (unsuccessful) approach to a “skeptic” blogger to post the link to the survey (the first one had been made two weeks earlier, at which time other “skeptic” bloggers were also contacted), the suggestion arose that because “skeptic” bloggers had been contacted late, the survey responses had been “decided” before the data had been received [DC 47]. This hypothesis implies that the data would have differed at a later point. Given that none of the “skeptic” blogs posted a link, and therefore could not have affected the result at any point in time, this hypothesis rests on a counterfactual assumption about the world.
A more extreme variant of this hypothesis proposed that the survey only gave the appearance of legitimacy to a pre-ordained result. [DC 30] This hypothesis identifies the survey as a “cover-up” for pre-ordained results that, presumably, were fabricated by LOG12: It thus goes a step beyond the hypothesis that a subset of the responses were “scammed.” These comments reveal an intense degree of suspicion (criterion Overriding Suspicion), an assumption of questionable motives by the LOG12 authors (Questionable Motives), and the belief that something must be wrong (Must be Wrong).
The “skeptic” blogs were contacted at least a week after the links to the study had already been posted on the eight other blogs that agreed to participate in the study. This delay did not go unnoticed by the blogosphere, with one blogger arguing that the delay represented conduct that fell short of “reputable” [DC 95]. The hypothesis never matured to the point of clarifying how this delay could have had any bearing on the outcome of the published LOG12 data, given that none of the “skeptic” blogs posted the link. The hypothesis therefore represents another instance of unreflexive counterfactual thinking, in addition to suspicion and the attribution of questionable motives (Questionable Motives, Overriding Suspicion, Must be Wrong). We also suggest that this hypothesis meets the criterion that “nothing is an accident” (Nothing is an Accident) because it imputes significance and intentionality into an event (i.e., a delayed email) that could equally have been accidental.
Because question order was counterbalanced between different versions of the LOG12 survey, links to the various versions were quasi-randomly assigned to participating blogs. The existence of different versions of the survey—in particular differences between the versions sent to pro-science and “skeptic” blogs—gave rise to several hypotheses, including the claim that “inconsistent sampling” invalidated the results of LOG12 [DC 65].
This hypothesis rests on counterfactual thinking: Even if survey versions had differed on some variable other than question order (they did not), given that none of the “skeptic” blogs posted the link and hence did not contribute responses, any claim regarding the published data based on those differences among versions rests on a counterfactual state of the world.
On 7 September, S.L. published a blog post explaining the reason for the different versions of the survey. (
Data collection for LOG12 also involved an attempt to recruit a “control” sample via an emailed invitation to participate in the survey among the first author’s campus community. Because this invitation returned only a small number of respondents (
When the survey invitation was discovered by a blogger, several questions emerged about those data, including whether a comparison with the blog sample had been conducted, and whether the data had then been discarded [DC 117]. Reflecting the pervasive belief that something must be wrong (Questionable Motives, Must be Wrong), those questions metamorphosed into the suggestion that the data reported by LOG12 were “cherry-picked” [DC 125].
Following standard Internet research protocols (e.g.,
Some members of the blogosphere interpreted this statement to mean that LOG12 accepted multiple responses provided they differed only slightly [DC 88]. Although this statement was initially qualified by noting that it was just an “interpretation”, this qualifier was dropped from subsequent re-posts of the allegation by other bloggers. The re-posts thus presented the unqualified claim that multiple responses from the same IP address could be included in the LOG12 data. The spread of this hypothesis despite being based on an interpretation alone reveals considerable suspicion (Overriding Suspicion) and also likely the belief that something had to be wrong (Must be Wrong). This theory lasted two days and was mentioned on a mainstream media blog in Australia, albeit without the qualifier that it rested only on an interpretation [DC 107].iv
On 14 September, the websites of the first two present authors (S.L.:
This gave rise to the claim by a blogger that both sites had specifically targeted his IP number to prevent access, based on the fact that this individual could gain access using an IP-anonymizing service. The blogger suggested that this prevention of access was unethical [DC 131]. This hypothesis is illustrative of the tendency to assign intentionality to random events (Nothing is an Accident), based on a background of a presumed questionable motive (Questionable Motives) and a self-perception as a victim (Persecution-Victimization).
The claim of IP blocking, although relatively minor in scope, is nonetheless notable because it escalated into a more intricate alleged plot by LOG12 to paint their critics as paranoid. Three comments are particularly indicative of this unfolding discourse:
One commenter predicted that the blogger’s IP number would be unblocked, to enable the LOG12 authors to level charges of paranoia against that blogger [DC 131].
Another commenter suggested that the blocking may have been an attack that was deniable and would leave no traces, enabling the LOG12 authors to respond with “told you so” if the blogger had complained [DC 131].
A final comment begrudingly applauded the skill (of the LOG12 team) to “play the audience” because no complaint (about the IP blocking) could be made without appearing to be a conspiracy theorist [DC 131].
The complexity of this argument parallels conspiracist reasoning surrounding the events of 9/11:
After 9/11, one complex of conspiracy theories involved American Airlines Flight 77, which hijackers crashed into the Pentagon. Even those conspiracists who were persuaded that the Flight 77 conspiracy theories were wrong folded that view into a larger conspiracy theory. The problem with the theory that no plane hit the Pentagon, they said, is that the theory was too transparently false, disproved by multiple witnesses and much physical evidence.
The blogosphere’s apparent concern over being “baited” into “acting paranoid” is consonant with the overriding extent of suspicion identified earlier as a criterion (Overriding Suspicion) of conspiracist ideation. It also reveals the pervasive self-perception of people who deny the scientific consensus on climate change as victims (Persecution-Victimization). The hypothesis also exemplifies the conspiracist tendency to detect meaning and intentionality behind accidental events (Nothing is an Accident).
The IP blocking hypothesis persisted for a day. The originator of the claim then updated his comment and removed the “unethical” charge, albeit without noting this update on his/her website. (The present authors had web-archived the initial version of the comment by then, which otherwise would have been irretrievably lost.) The originator of the claim then publicly recognized that the blocking might have reflected technical glitches in the link to Australia [DC 131].
Some commenters expressed general displeasure with LOG12, for example by referring to it as an “egregious war crime” akin to “shooting innocent men” and then using those numbers as “terrorists killed in action” (DC 28). Two miscellaneous hypotheses deserve particular mention as they provide insight into the recursive and self-reinforcing nature of the blogosphere’s discourse.
A regular contributor to the blog of the second author of the present paper (J.C.;
A further hypothesis supposed that the real purpose of LOG12 was to provoke conspiracist ideation from people who reject climate science, asserting that the subject of the study was the blogosphere’s response to LOG12 rather than the survey itself. The commenter adduced support for this assertion from private correspondence of the second author’s blog (J.C.;
At the time of this writing, similar hypotheses have circulated regarding a Massive Open Online Course (MOOC) run by the second author (J.C.) at the University of Queensland. This MOOC is entitled “Making Sense of Climate Science Denial” (
Thus far, we considered only strictly recursive theories—that is, hypotheses that were spawned by LOG12 and pertained to the methodology and results of LOG12. We conclude with an analysis of theories that were spawned by LOG12 but expanded beyond being recursive.
The expansion commenced with the suggestion that the University of Western Australia was hosting an “activist organization.” It furthermore questioned whether UWA Executives were aware of this global “activism” [DC 110].
Another blogger promoted this theory, linking to the above post [DC 110] and commenting that the blog of the present second author was a ringleader for “conspiratorial activities” [DC 136]. Notably, this blogger explicitly referred to
A commenter sought to clarify the extent of this presumed conspiratorial activity, identifying another academic at UWA, Professor of Mathematics X, as the “real strategist” behind those activities [DC 111]. X’s apparent leadership role in this conspiracy was reinforced in a subsequent comment by a local who confirmed, with “no doubt”, X’s role as “mastermind” and typical “mad scientist” [DC 112].
A more extended variant of this hypothesis cited S.L.’s research funding available on his webpage (A$4.4 million in grants) and drew attention to A$762,000 specifically for climate research. Moreover, the commenter argued that this funding did not include A$6 million that the Australian Commonwealth Government provided S.L. and colleagues to run ‘The Conversation’ [DC 122]. ‘The Conversation’ refers to an online newspaper (
The expanding scope of the presumed activities exhibited considerable longevity, as evidenced by a blogpost several months later that was triggered by a radio interview with S.L. on the Australian Broadcasting Corporation’s (ABC) science show. This blogpost accused the (Australian) government of suppressing dissent and labeled S.L.’s research as representing “punitive psychology”, akin to the Soviet Union’s practice to incarcerate dissidents in mental institutions. The blogpost argued that the problem of censorship is widespread and involved the University of Western Australia, the Australian Research Council, the Australian Broadcasting Corporation, and possibly the government.
Common to all these hypotheses is the presumption of widespread questionable motives among the authors of LOG12 and colleagues (Questionable Motives) and a potentially self-sealing propensity to broaden the scope of the presumed malfeasance (Self-Sealing). In this instance, extending the presumed malfeasance to include the Australian government of the day may amplify a self-perception of being victimized (Persecution-Victimization).
On 10 October 2012, the host institution of S.L. released a tranche of emails and documents that had been requested by a “skeptic” blogger under Freedom-of-information (FOI) legislation. One set of emails involved all correspondence between the researchers and the blogs that were contacted to host the survey, including those that by an initial hypothesis—number 2 in
The blogosphere focused on the ethics approvals underlying the study. The existence of ethics approval was met by a broadening of the scope of presumed malfeasance, from the authors of LOG12 to the ethics committee and its chair. To illustrate, one blogger cited the fact that approval was obtained by amendment, and that the speed of approval (within 24 hours) raised questions about the university, especially because the original ethics approval had ostensibly been for a fundamentally different project [DC 165]. In fact, the approval of the additional items for this particular study, drawn from previously published research (e.g.,
The thematic analysis provides an initial sketch of the blogosphere’s collective response to LOG12. The contribution of Study 1 was to classify the blogosphere’s discourse into a number of potentially conspiracist hypotheses on the basis of a number of well-established criteria. This result meshes well with the prior literature, which has repeatedly linked the denial of well-established scientific findings to conspiracist ideation (e.g.,
However, Study 1 has two limitations that need to be addressed before we can explore the full implications of the results. First, any thematic analysis necessarily contains a subjective element, which opens the door to alternative interpretations. Second, this problem is compounded by the fact that two of the present authors also contributed to LOG12, thereby creating a potential conflict of interest.
We addressed these problems in a number of ways: First, the data collection for Study 1 (via Internet search) was conducted by two of the present authors who were not involved in analysis or report of LOG12. Second, the fourth and fifth authors (S.B. and E.A.L.), who had no involvement with LOG12 whatsoever, provided an independent audit of Study 1, reviewing the data and the categorization of it presented in
Third, and arguably most important, we now report two behavioral studies that sought independent support for the thematic analysis using naive participants in a blind test. For those studies, an anonymized and enlarged set of comments was harvested off the Internet in a further search conducted by hired research staff with no association with the present authors.
In Study 2, naive subjects were presented with the material underlying the thematic analysis in Study 1 and were asked to indicate for each item whether it represented one of a broad range of potential recursive hypotheses. The thematic analysis in Study 1 would be supported if the final classifications were to recreate a variant of
The material for the thematic analysis in Study 1 had been obtained using a “depth” approach (e.g., by following up on mentions of a hypothesis by searching targeted sites for recurrences or derivatives), which required some skill and discretion and thus had to be conducted by two of the authors. For the remaining studies, an additional “breadth” search of the Internet was conducted by a contracted assistant. The assistant was not academically associated with any of the authors although he was a PhD student at UWA at a time when S.L. was at the same institution (a year earlier).
This Google search was extended to cover the period 8 July 2012 to 8 October 2012, using a daily search of blog content and the search phrase “Stephan Lewandowsky”. Unlike for the thematic analysis, only the first 100 comments of any applicable post were scanned and no follow-up “depth” search was performed. Content was included if it was minimally recursive (i.e., pertaining to LOG12). This search initially yielded 429 content items.
Those content items were then compared with the items from the “depth” search from Study 1, using an iterative matching algorithm as follows: Random 50-character strings were selected from each “depth” content item and compared to the “breadth” content set for matches. This process was repeated using randomly-chosen 50-character strings for a given content item until no further matches were located. The strings were 50 characters in length because using shorter strings resulted in false matches. The final search using 50 characters strings matched 40 content items from the “depth” set to the “breadth” set. Each match was verified by human inspection.vi In consequence, there were 131 items in the “depth” set from Study 1 that were not returned in the “breadth” search in Study 2.
Results from the two searches (i.e., all of the “breadth” items plus the unique items from the “depth” set) were concatenated into a corpus of content material for use in Study 2 and Study 3. Because the combined corpus was prohibitively long and some content excessively verbose, a research assistant eliminated content that was deemed to constitute duplications or was deemed to be too diffuse or incomprehensible to merit inclusion as stimuli. Content items were also edited to reduce their length (by replacing non-essential content with “...”), where necessary. After editing, which removed less than 10% of content, the final set comprised 508 content items, of which 382 were from the “breadth” search (inclusive of items that were also returned by the “depth” search). A further 126 content items were from the “depth” search for Study 1. The material was then anonymized by replacing the names of individuals with placemarkers such as “Author 1” (for S.L.) or “Paper” (for LOG12) and so on. After editing, the longest comment contained 223 words (
The stimulus material was then presented to two research assistants who were asked to jointly produce a broad and (ideally) exhaustive set of potential recursive hypotheses for use as a scoring sheet in Study 2. The intention was to capture all hypotheses across the entire corpus at a fine level of detail, for subsequent scoring of the content material by another set of blind judges.
The final scoring sheet contained 38 candidate hypotheses, which subsumed hypotheses 1 through 7 from
The participants were 5 psychology undergraduates in their Honours (4th) year of study at the University of Western Australia who were not known to any of the authors and were unaware of LOG12 or the purpose of the study. At the time the study was conducted (late in the academic year), Honours students would have completed their independent research project and would have submitted (or be close to submission of) their final thesis. Australian Honours students are comparable in competence and background training to junior PhD students in the U.K. and (often) the U.S. The participants were thus competent to conduct the rating task. Participants received A$15 per hour in exchange for participation. Experimental sessions took 3-5 hours (with self-paced breaks as required).
Participants were presented with the corpus of 508 content items in a spreadsheet and were informed that they “relate to a scientific paper written by several anonymized authors, identified as Author 1, etc.”. Participants classified each content item into one of the 38 candidate hypotheses provided on a printout of the score sheet, by entering the corresponding hypothesis number into a column in the spreadsheet. Participants could indicate that no hypothesis applied by placing a zero in the response column. Participants had the option to nominate another novel hypothesis by writing a brief description into the response column (no participant exercised this option). Participants had to record at least one response for each item (i.e., a 0 or their preferred hypothesis number), but they could optionally add a second and third classification in two more response columns for each content item if they thought that a comment referred to several hypotheses.
We first asked what proportion of content items participants did
We next examined the consistency of those classifications (including the “other” categories) across participants. For this analysis, we considered the mandatory primary classifications and the secondary classifications where present (.28 of all cases). The small proportion of optional tertiary (.04) classifications were not considered. Because the instructions to participants did not differentiate multiple classifications by importance (i.e., participants were instructed to use “up to 3” classifications if they thought a “comment refers to several hypotheses”), primary and secondary classifications were considered interchangeably. For example, if for a given comment, 4 participants provided a primary classification into hypothesis X, and the fifth participant’s primary classification was Y but that person’s secondary classification was X, then this was counted as agreement among 5 participants.vii
Number of Identical Classificationsa |
||||||
---|---|---|---|---|---|---|
Set of items | 1 | 2 | 3 | 4 | 5 | 3 or more |
Combined | 20 (.04) | 166 (.33) | 147 (.29) | 115 (.23) | 60 (.12) | 322 (.63) |
“Depth” (Study 1) | 2 (.02) | 38 (.30) | 38 (.30) | 32 (.25) | 16 (.13) | 86 (.68) |
“Breadth” (Study 2) | 18 (.05) | 128 (.34) | 109 (.29) | 83 (.22) | 44 (.12) | 236 (.62) |
aCell entries refer to the number of comments (proportion in parentheses) that were classified identically by the number of judges in that column.
Number of identical classificationsa |
||||||
---|---|---|---|---|---|---|
Set of items | 1 | 2 | 3 | 4 | 5 | 3 or more |
Combined | 2 (.003) | 62 (.12) | 132 (.26) | 138 (.27) | 174 (.34) | 444 (.87) |
“Depth” (Study 1) | 0 (.000) | 11 (.09) | 37 (.29) | 31 (.25) | 47 (.37) | 115 (.91) |
“Breadth” (Study 2) | 2 (.005) | 51 (.13) | 95 (.25) | 107 (.28) | 127 (.33) | 329 (.86) |
aCell entries refer to the number of comments (proportion in parentheses) that were classified identically by the number of judges in that column.
We next asked whether the principal hypotheses from
IDa | Nb | Summary of hypothesis |
---|---|---|
4 | 16 | |
2 | 12 | |
31 | 6 | |
6 | 6 | Other sampling |
7 | 4 | |
20 | 4 | Research methodology was flawed |
3 | 3 | |
5 | 3 | |
18 | 3 | Author 1 created Paper 1 to pathologise “skeptic” |
19 | 3 | |
21 | 3 | Statistical analysis was flawed |
37 | 3 | Author 1 was/is biased |
38 | 3 | Other ethical flaws |
1 | 2 | The sample size was insufficient (e.g., too small) |
12 | 2 | Author 1 colluded with other scientists |
29 | 2 | Conclusions were misrepresented or misinterpreted |
30 | 1 | |
36 | 1 | Author 1 faked the data |
aCell entries refer to the ID number of a hypothesis in the scoring sheet used in Study 2. Descriptions of hypotheses are abbreviated to permit cross-referencing with
bCell entries are the number of comments consistently assigned to that hypothesis by 3 or more participants.
Further statistical confirmation for this independent recovery is provided by correlating the number of comments assigned to each of the principal hypotheses from the thematic analysis by the present participants (bold-faced in
A final question concerns potential differences between the two search sets (i.e., the “depth” set used in Study 1 and the “breadth” set generated for this study).
IDa | Nb | Summary of Hypothesis |
---|---|---|
4 | 27 | |
2 | 23 | |
20 | 23 | Research methodology was flawed |
1 | 15 | The sample size was insufficient (e.g., too small) |
6 | 15 | Other sampling |
14 | 14 | Peer review was biased |
21 | 12 | Statistical analysis was flawed |
9 | 10 | Questionnaire poorly constructed |
7, 10 | 8, 5 | |
36 | 8 | Author 1 faked the data |
8 | 6 | Absence of neutral response option |
28, 29 | 6, 4 | Conclusions were misrepresented or misinterpreted |
37 | 6 | Author 1 was/is biased |
15 | 5 | Paper politically motivated |
33, 30 | 4, 5 | |
5 | 4 | |
25 | 3 | Poor operationalization |
26 | 3 | Method improperly/insufficiently reported |
23 | 2 | Literature review flawed |
34 | 2 | Author attempts to silence debate |
38 | 2 | Other ethical flaws |
11 | 1 | Other problem with questionnaire |
12 | 1 | Author 1 colluded with other scientists |
16 | 1 | There is an ulterior motive or hidden agenda to the paper |
17 | 1 | Author colluded with journalists |
22 | 1 | Research question flawed |
31 | 1 |
aCell entries refer to the ID number of a hypothesis in the scoring sheet used in Study 2. Descriptions of hypotheses are abbreviated in this table to permit cross-referencing with
bCell entries are the number of comments consistently assigned to that hypothesis by 3 or more participants.
The frequency information in
Study 2 provided independent support for the classification of hypotheses and their frequency that formed the basis of our thematic analysis in Study 1. Because Study 2 compared content material from two different search sets, the results suggest that the classifications used in the thematic analysis were not contingent on a particular search strategy.
Critics might argue, however, that the present results were biased by the fact that participants were provided with a fixed set of choices (i.e., the 38 hypotheses on the scoring sheet), and that the emergence of the hypotheses in
A further criticism might point to the fact that Study 2 yielded several additional hypotheses (see
In Study 3, participants were presented with a set of content items from two sources: A sample from the combined corpus used in Study 2 and a comparison sample that was generated by academically-trained critics. Participants provided subjective ratings for each item on a number of dimensions that were designed to tap the criteria for conspiracist ideation in
The stimuli for Study 3 comprised the corpus of 508 content items used in Study 2 and an additional set of 43 comparison items that were intended to represent incisive and scientifically-argued criticisms of LOG12. The comparison items were generated by 3 PhD students in Psychology at the University of Bristol (not known to any of the authors at the time; S.L. had only moved to Bristol a few months earlier and was not yet teaching or supervising students). The students were provided with a copy of LOG12 and the instructions to “(...) generate comments about this paper that could be posted on Internet blogs.
The web portal constrained the word length of comments by calculating a running average word length of all submitted comments up to that point. If the average word count was greater than 65 (i.e., the average word count of the corpus from Study 2), then the maximum word count allowed for a submitted comment was constrained to 265 minus the average word count. If the average word count was less than 65, then the maximum word count allowed was set to 200. The final set of 43 comments were roughly comparable in length (
The participants were 25 members of campus communities: 5 were from the University of Western Australia (mainly undergraduate students) and 20 were psychology undergraduates at the University of Bristol who participated voluntarily in exchange for course credit or remuneration at the rate of A$10 or £10 per hour, respectively. The sample comprised 16 female and 9 male participants with an average age of 24.68 years (
The experiment was controlled by a Windows computer that presented all stimuli and recorded all responses with the aid of the Psychtoolbox for MATLAB (
The two types of items were randomly intermixed into a single sequence of 86 rating trials. On each trial, the content item was presented at the top of the screen, and participants then responded to five test questions using the numbers 0 through 9 to indicate their judgment. Each test question was presented below the content item and replaced the previous test question (if any).
Itema | Short Labelb |
---|---|
…believe that the scientists acted with questionable motives? | Questionable Motives |
…express deep-seated suspicion? | Suspicion |
…perceive himself/herself as a victim of scientists or research? | Victim |
…firmly believe that there must be something wrong with the research? | Something must be Wrong |
…offer a reasonable and well thought out criticism of the research? | Reasonable Critique |
aEach test question combined the phrase “To what extent does the commenter…” with the text given in the table entry.
bShort label used in figures and to refer to a measure in the text.
Participants responded at their own pace and each test question remained visible until a response was made. Trials were separated by a 2-second period during which the screen went blank.
Due to equipment failure, data from 3 subjects were missing responses to the final 11, 5, and 37 content items, respectively. All subjects were retained for analysis (the results do not change appreciably if participants with missing observations are omitted). Responses were averaged across trials separately for each content type to yield two data points per subject (one for
Five independent analyses of variance revealed significant main effects of comment type for all measures. The test statistics are shown in
Dependent variable | MSE | PhD = 9 | Web = 9 | ||
---|---|---|---|---|---|
Questionable motives | 72.14 | 1.00 | <.0001 | .042 | .266 |
Suspicion | 101.60 | 0.81 | <.0001 | .041 | .259 |
Victim | 66.65 | 0.64 | <.0001 | .011 | .096 |
Something must be wrong | 62.39 | 0.56 | <.0001 | .050 | .353 |
Reasonable critique | 29.53 | 0.97 | <.0001 | .077 | .030 |
Summary of overall responses for all 5 measures in Study 3. Thick horizontal lines are medians, and medians differ significantly if the notches between boxes in each panel do not overlap. See
The last two columns in
A visual illustration of the skew in the response distributions for web content is shown in
Histograms (dark gray) for all responses to web content (not averaged across trials within a participant) for all 5 questions in Study 3. The dashed line that encloses the cross-hatched area represents Gaussian kernel density estimates (using 1.5 times the standard bandwidth) of the equivalent responses to PhD-generated content.
The results of Study 3 are straightforward: Compared to material generated by junior scholars who were instructed to be as critical as possible of LOG12, the discourse in the blogosphere scored higher on the four criteria for conspiracist ideation tested in this study. One potential criticism of this result is that the junior scholars—notwithstanding the promise of anonymity and notwithstanding the fact that they were not known to any of the present authors—might have been constrained by professional courtesy and therefore might have attenuated the full extent of their critique. This criticism can be deflected on the basis of two aspects of our results: First, participants judged the PhD-generated material to offer a more thought-out criticism than the responses of the blogosphere. This significant difference is difficult to reconcile with attenuation (and could only be even greater if any possible attenuation were absent). Second, the fact that a large proportion of blogosphere material was given an extreme rating on most of the potentially conspiracist attributes is diagnostic by itself, even in the absence of any comparison with the PhD-generated material. The web-content items thus largely spoke for themselves and although the PhD-generated material provided comparative context, it was not essential to our conclusions.
Our results were obtained with anonymized material that was presented with no guiding context, to participants who knew nothing about the issue under consideration. Moreover, the blog comments included content from both Internet searches conducted for this project (i.e., the “depth” search for Study 1 and the “breadth” search for Study 2), and the two different search sets were found to make no difference to people’s ratings. The data thus again support the results of our thematic analysis in Study 1.
The present studies were concerned with the blogosphere’s response to a single 4,000-word article. One might therefore question the generality of our results. In response, we note that at least one other scientific report in the climate arena engendered a sustained critique that subsequent scholarly analysis identified as conspiracist (
A second criticism might cite the fact that in our thematic analyis we have considered the “blogosphere” as if it were a single entity, analyzed within the context of psychological processes and constructs that typically characterize the behavior of individuals rather than groups. Our response is twofold: First, at the level of purely descriptive narrative methods, our work fits within established precedents involving the examination of communications from heterogeneous entities such as the U.S. Government (
A further criticism might hold that although we may have presented some evidence for the presence of conspiracist ideation, the evidence falls far short of “real” conspiracy theories involving events such as 9/11 or the moon landing. In response, we note that the hypotheses leveled against LOG12 do not differ qualitatively—that is, in terms of magnitude or scope—from others that have been identified as conspiracist in the context of another paper in the climate arena (
Critics might further invoke the fact that two of the present authors also authored LOG12, thereby creating a potential adverse conflict of interest. On this view, the response of the blogosphere represented legitimate criticism of LOG12, and the present article is merely an attempt to deflect the impact of that criticism. We argue against this view on multiple grounds. First, the present article fits squarely with precedents in the scholarly literature of researchers analyzing or reporting events arising from their own work (e.g.,
It must also be noted that the present article arguably goes against the interests of the LOG12 authors because it places several criticisms of LOG12 into the peer-reviewed literature that previously had been limited to Internet blogs. Given the well-known resistance of information to subsequent correction (e.g.,
Bearing in mind those potential limitations, we now explore the two goals stated at the outset. We explore the implications of our work for understanding of conspiracist ideation and for the role of the blogosphere, and the public more generally, in the conduct of science.
Our principal thesis is that some of the responses to LOG12 voiced in the blogosphere satisfy attributes of conspiracist ideation by the criteria defined at the outset. Two attributes deserve to be highlighted: First, most of the hypotheses can be unified under the immutable belief that “there must be something wrong” (Must be Wrong; see
Whereas suspicion on its own is insufficient to identify conspiracist ideation, it arguably constitutes one of its core attributes. For example, the suspicion that LOG12 did not contact “skeptic” bloggers tacitly invokes several major presumptions, namely (a) that the authors of LOG12 were willing to engage in research misconduct; (b) that they would invent a claim about a non-event and publish it in a Method section when there was no incentive or reason to do so; and (c) that they should have somehow provided more “evidence” for the method they used beyond writing an accurate Method section.
Indeed, most of the hypotheses advanced about LOG12 included an accusation of intentional wrong-doing by the authors (viz. minimally hypotheses 2, 4, 5, 6, and 8), which goes beyond pointing to problems and errors as would be the norm in conventional scientific discourse (and as was empirically confirmed in Study 3). The ease with which those presumptions about misconduct and malfeasance were made and accepted provides a fertile environment in the blogosphere for the subsequent unfolding of conspiracist ideation (cf.
Those underlying beliefs infused conspiracist elements even into those hypotheses that would be expected to arise during routine scholarly critique. For example, the scamming hypothesis evolved continuously without being guided by clear a priori assumptions about what would constitute a scammed response profile, thereby ultimately rendering this hypothesis self-sealing and unfalsifiable (criterion Self-Sealing). It is this psychological attribute, combined with the lack of clear a priori standards for a profile for a “fake” response, that points towards a conspiracist component rather than conventional scholarly critique.
Second, self-sealing reasoning also became apparent in the broadening of the scope of presumed malfeasance on several occasions. When ethics approvals became public in response to an FOI request, the presumption of malfeasance was broadened from the authors of LOG12 to include university executives and the university’s ethics committee. Similarly, the response of the blogosphere evolved from an initial tight focus on LOG12 into an increasingly broader scope. Ultimately, the LOG12 authors were associated with global activism, a A$6 million media initiative, and government censorship of dissent, thereby arguably connecting the response to LOG12 to the grand over-arching theory that “climate change is a hoax.”
Notably, even that grand “hoax” theory is occasionally thought to be subordinate to an even grander theory: one of the bloggers involved in the response to LOG12 considers climate change to be only the second biggest scam in history. The top-ranking scam is seen to be modern currency, dismissed as “government money” because it is not linked to the gold standard. The observed broadening of scope meshes well with previous research that has identified stable personality characteristics that predict the propensity for conspiracist ideation (cf.
Our research points to at least two issues that merit further investigation. The first issue concerns the prevalence of the discourse we have documented in this article.
Our studies provide an “existence proof ” for conspiracist discourse, and the “breadth” corpus also contains quantitative information suggesting that this element of discourse was quite common in this instance. Nonetheless, extrapolation from our circumscribed corpus to the overall prevalence of conspiracist discourse in the climate blogosphere is fraught with risk and may be inadvisable. There is, however, other recent evidence that establishes the prevalence of conspiracist discourse in climate denial generally (as well as in the rejection of other scientific propositions). For example, the content on “skeptic” climate blogs that is dedicated to the “climategate” pseudo-scandal has been steadily increasing since 2009 (
More direct evidence for the prevalence of conspiracist themes relating to climate change on social media was provided by
Outside the Anglosphere,
Second, we uncovered a potentially novel aspect of conspiracist reasoning when some of the later hypotheses were found to involve a residual impact of earlier, discarded hypotheses. For example, whereas critics initially argued that the results of LOG12 were invalid because “skeptic” bloggers were not contacted (hypothesis 2 in
This point requires further analysis, because counterfactual reasoning—that is, the use of contrary-to-fact premises in arguments against explanatory hypotheses—is common in legitimate scholarly critiques. This reasoning was indeed present in any assertion that if the skeptic sites had posted links to the survey, then the LOG12 results would have been different. Irrespective of the truth or falsity of the subjunctive counterfactual conditional that the data would have been significantly changed (which is an open empirical question, adjudicated only partially by the replication reported in LGO13), this use of counterfactual reasoning legitimately raises the question of whether the LOG12 data are based on a sufficiently representative sample.
By contrast, the unreflexive counterfactuals that assert skeptic sites were not contacted or were contacted only after a delay (as reported in the context of Study 1), are striking for two reasons. First, even if the counterfactuals were true (as in the case of hypothesis 4; viz. the delay in contacting “skeptic” blogs), they nevertheless have no implications about how the survey results might have been different. Unless links to the survey had been posted on skeptic sites eventually, irrespective of any delay, the unreflexively asserted counterfactuals about delays (or the false notion that “skeptic” blogs were not contacted) would have no bearing at all on the accuracy of the survey results. In short, even if the counterfactuals are granted as premises, they would not imply the conclusion that the results are skewed in some way because the results as reported do not depend on a non-existent state of the world.
There is, however, a way to interpret these comments that would make the counterfactuals less irrelevant. Perhaps they should be interpreted as asserting that “Had the ‘skeptic’ sites been contacted in a different manner, they would have posted a link to the survey and their readers would have responded in ways that would have changed the results (and made them more accurate).” Intriguingly, there is no evidence that these unreflexive counterfactuals were intended to be offered in support of that sort of claim. Instead, they seem to be offered either in support of the questionable motives of the investigators, that something is wrong, or that the “skeptics’ ” suspicions would lead them, as some explicitly asserted (see Study 1), to in fact do the opposite, to not post a link to the survey. We therefore suggest that this unreflexive counterfactual reasoning may be part of the toolbox for conspiracist discourse and may warrant further scholarly attention.
Could the activity in the blogosphere have constituted legitimate scholarly critique of LOG12, rather than a partially conspiracist discourse as is argued here? Several considerations speak against that possibility. First, the data from Study 3 show that unbiased observers were unable to discern much scholarly value in the blogosphere’s response to LOG12.
Second, we already noted at the outset that the public discourse in the blogosphere was accompanied by non-public events to prevent or delay the publication of LOG12. The attempts to suppress publication of a peer-reviewed paper, in conjunction with the absence of any discernable conventional scholarly activity, speak against the possibility that the blogosphere’s discourse represents legitimate public engagement with science. There are indications that seeking the retraction of inconvenient papers has become a routine practice among individuals who are denying (climate) science: We already noted the circumstances surrounding Recursive Fury at the outset. Moreover, one of the bloggers who was also involved in the response to LOG12 recently called for the retraction of a peer-reviewed paper that had underscored the pervasive scientific consensus on climate change through an analysis of nearly 12,000 peer-reviewed articles (
The present findings add to the body of evidence showing that blog posts and comments sometimes have adverse effects on the rationality and civility of public discourse. For example,
In response to those risks, at the time of this writing, several large online news services, such as Popular Science (
Our results therefore contribute to a growing understanding of online behavior and its implications for the conduct of science and public discourse. We offer two tentative pointers towards possible solutions. First, it is increasingly clear that there is a need to provide better architectures for online platforms to help channel public discourse into a more constructive direction. The requirement to back up claims by evidence (as on Reddit) or to consider factual status in search engines (as recently piloted by Google) point in that direction. Other alternatives include strict moderation of comments, as for example practiced by the online newspaper
Second, our results point to the need to educate the public about the difference between scientific and non-scientific forms of discourse. The Internet—as a platform for everyone to voice any opinion and make any claim, however unsupported by evidence—will not go away, and the positives associated with a “free for all” medium should not be under-estimated. However, we need to protect the evidence-bound sphere of scientific arguments from the largely unconstrained buzz outside that sphere. Peer review certainly has a role in defending that boundary: As far as we know, none of the individuals involved in the blogosphere’s response to LOG12 ever submitted a critique for peer review. Conversely, a peer-reviewed critique of LOG12 and LGO13 has recently appeared in print (
The first author was supported by a Discovery Outstanding Researcher Award from the Australian Research Council during part of this research, and he has been supported by a Wolfson Research Merit Award from the Royal Society since 2013. In addition, the research was supported by internal funding from the University of Bristol and the University of Western Australia. The remaining authors have no funding to report.
The authors have declared that no competing interests exist.
The authors thank Charles Hanich for assistance throughout the project, and Alexandra Freund for comments on an earlier version of the manuscript. S.L. blogs at
In current scholarly usage the term “denial” is often reserved to describe an active public denial of scientific facts by various means, such as the use of rhetoric to create the appearance of debate where there is none (
The criteria for this hypothesis may also have shifted in response to a blogpost by two of the authors of LOG12, which demonstrated the resilience of their main findings to the removal of outliers on the measure of greatest interest, viz. the endorsement of the various conspiracy theories, on 12 September 2012 (
This statement was made on the same day that the bloggers’ names were released and it is impossible to ascertain whether it predated or postdated the release.
Following a complaint to the Australian Press Council by S.L., this post was subsequently amended (in 2013).
At the time of this writing, this initial FOI request had been followed by at least 4 more FOI requests for items ranging from correspondence to the publication dates of blog posts.
It was necessary to deploy this fairly complex matching algorithm because the length of content items sometimes differed between the two sets; that is, the “depth” set might have contained a shorter excerpt of the same comment or blog post than the “breadth” set, or vice versa. Given the strict criteria for a match, which avoided false positives altogether, the number of matches was likely an underestimate.
The results are qualitatively identical if only primary classifications are considered.
Note that the PhD items included numerous content items that were generated by the same person, which might introduce a correlation among responses to those items. This is less likely for the web items which were sampled from a much larger set (although the overall corpus likely included multiple items produced by the same person). Any potential dependence among responses for each category would, however, have no statistical consequence because all responses were averaged within each content type, and hence each participant only provided a single observation for the analysis for each type. Those responses are independent across participants, as required by the analysis of variance.
This skew could not be detected in the boxplots in
It must be noted that the distribution of contributions to the blogosphere’s discourse is far from even. The search string “<firstname><lastname>” Lewandowsky (with actual names replacing the <> placemarkers) yields between 5,000 and 9,000 Google hits for 4 particularly active individuals.