In late 2012 there was a lot of adverse comment about the paper Lewandowsky, Oberauer & Gignac – NASA faked the moon landing:Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science (in press, Psychological Science). I did my own quick analysis using pivot tables, which was referred to elsewhere.
Last week, Brandon Shollenberger produced a shorter survey that, though tongue in cheek, aimed to replicate the findings of the Lewandowsky et al. He wrote
As you’re aware, Stephan Lewandowsky has written several papers claiming to have found certain traits amongst global warming skeptics. I believe his methodology is fundamentally flawed. I believe a flaw present in his methodology is also present in the work of many others.
To test my belief, I’m seeking participants for a short survey (13 questions). The questions are designed specifically to test a key aspect of Lewandowsky’s methodology. The results won’t be published in any scientific journal, but I’ll do a writeup on them once the survey is closed and share it online.
This was published at the blogs Wattsupwiththat, JoanneNova and BishopHill blogs. The poll is still available to view.
A few hours ago Jo Nova published Shollenberger’s initial findings, as “Warmists Are Never Wrong, Even When Supporting Genocide“. Using the same methodology that Lewandowsky et al (LOG12) “demonstrated” that those who reject the climate religion have a propensity to believe in cranky conspiracy theories, Shollenberger showed that believers in catastrophic global warming have a propensity to believe in genocide, paedophilia and human trafficking. Like for the LOG12, I have run the data through Excel pivot tables to reveal that Shollenberger was successful in undermining LOG12.
Categorizing the responses
For the LOG12 I split the respondents according to the average response to the four LOG12 “climate science” questions.
Similarly, with the Shollenberger survey, I have categorised the respondents according to response to the three questions on global warming. This time I weighted the responses in relation to belief in catastrophic anthropogenic global warming. First I changed the 1 to 5 response to a 0 to 4 response. The weightings were then 1 for Ques 1, 2 for Ques 2 and 4 for Ques 3. By dividing by the maximum score of 28, I obtained a “believer” percentage. Questions are below.
Also, I have looked at the percentage with the outlier scores, along with the average scores.
Preliminary observations
Some brief preliminary observations that stand out from the pivot tables. These are the green bordered summaries below and the responses to the individual questions at the foot.
- Compared with LOG12, Schollenberger gets three times the responses and takes a week rather than 18 months to publish the results.
- Schollenberger shows the result of only publishing a survey on only one side of the global warming divide, whilst trying to analyse the other side. The vast majority of responses are from people you are not targeting.
- The three times response, in a much shorter time frame indicates that sceptics are far more interested in the subject of global warming than the believers.
- Proportionately, more far sceptics seem to visit “believer” blogs than “believers” visit sceptic blogs. This should not be controversial. Sceptics look to understand the view they oppose, whilst “believers” look for confirmation. Climate change is no different from many other areas, including many of the softer sciences.
- Schollenberger, in his three questions on belief in global warming captures a broader possible range of beliefs in the climate science, than LOG12 does in four questions. In particular it is possible to distinguish between those who believe humans have caused most of the recent warming, but it is fairly trivial, and those who (like the MSM) believes we are all doomed unless we abandon out cars for bicycles and go to 2W lightbulbs everywhere. The LOG12 questions were designed to polarize views into “pro-science” and “deniers”. Schollenberger thus achieves very quickly what millions of dollars spent on opinion surveys conceals. The extreme alarmism that justifies policy is not held by the majority who believe that anthropogenic global warming is an issue.
- Both surveys were uncontrolled for “scam” responses. That is for those on one side to be able to mischievously post as an opponent, but with reprehensible views. The Schollenberger survey had more, and (to a lesser extent) a higher proportion of scam responses. Given the knowledge of LOG12, this is not surprising. But, given the proportions of non-scam responses, “believers” seem to have a greater propensity to scam “sceptics” than the opposite.
- Thus Schollenberger can demonstrate that Lewandowsky’s conclusions are as much based on scam responses as his survey.
The Survey Questions
Number of Responses to questions 4 to 13, in relation to CAGW score.
ManicBeancounter 
bshollenberger
/ 16/01/2014Some thoughts:
1) Lewandowsky discussed his results before his survey even closed. I can’t fault him for taking 18 months to publish since getting published in peer reviewed literature may have been difficult. I can fault the peer review process though. It’s ridiculous such a simple project should take over a year.
3) My survey was more entertaining and was described in a more appealing way. Plus, it was shorter. I’m sure those things contributed.
4) I’m pretty sure you meant to type “skeptic blogs” at the end. I wouldn’t generalize like this though. The majority of the responses came from WUWT. It may just be that site they avoid.
6) It’s also worth considering other patterns which indicate fake responses. Also, I wouldn’t necessarily blame “believers.” The survey was silly. Other people may have scammed it too.
7) Scam responses actually aren’t entirely responsible for the correlations I got with this survey. There’s a methodological issue which practically guarantees certain results. I’m writing a post about it as we speak. From the testing I’ve done, it appears removing scam responses still results in 82% of the correlation pairs being statistically significant. That’s as opposed to 92% with them included.
It looks good though. I’m curious about one thing. Why did you scale by 1/2/4 instead of 1/2/3?
manicbeancounter
/ 16/01/2014Thanks Brandon for your comments. You are far more generous than I in not impugning false motives, and more willing to consider alternative explanations.
1) On the time Lewandowsky took to get over the hurdle of peer review, I am sure that is a factor. Another is that there is considerable background on showing the consistency of the results with other papers. But, given that an opinion poll did not give anything like the results of normal opinion polls, much effort was given to massaging the message. This included not revealing the small proportion of those taking the survey who were sceptics.
4) Thanks. Now fixed.
7) I should have referred a comment you made at Steve McIntyre’s Climate Audit, explaining, in very straightforward terms, how to get a positive correlation between two propositions when nobody actually agrees with both. In particular,
The same goes for any two pairs of statements that people are nearly unanimous in rejecting.
Why did you scale by 1/2/4 instead of 1/2/3?
I was looking at beliefs. I think there is plenty of evidence to show global warming in the last 150 years. There is also a loose fit with the rise in greenhouse gases, and there is a lot of support for the idea. But, belief in global warming as a serious threat requires belief in two propositions. First that the warming will accelerate. Second, that it will have huge adverse consequences. This despite considerable evidence of a pause, and the failure of all the short-term catastrophes. 1/2/4 thus seems a more reasonable approach. Also in classifying I noticed a number with low scores for the first two, but high scores for the serious threat, hence the group “5. Few signs, but the apocalypse is coming“. They have an opinion more based on belief than somebody who scores 3,3,3.
Brandon Shollenberger
/ 16/01/2014Glad to. It’s always nice to see discussion of what I write.
What you say for 7 is actually what I’m troubled by the most. Fake responses are an inherent risk with surveys, but this is a methodological problem which allows one to create spurious results on a whim. It’s obscene. Whether its impact is bigger or smaller than that of fake responses, it’s nature is far more troubling.
On the 1/2/4 scaling. I had thought you were just trying to emphasize the fake responses to highlight them (since the first two questions mattered less for the fake responses). Your explanation makes sense though. The main issue in the global warming debate is whether or not global warming poses a serious threat. A response to it tells us more than one to the other questions.
By the way, that comment was originally from an e-mail to Michael Wood. If you haven’t looked at his paper, you should. It’s hilarious how bad it is. Wood asked a bunch of people if they believed in a variety of conspiracy theories, including some contradictory ones. He found a positive correlation across them. From that, he concluded conspiracy theorists believe in contradictory conspiracies. It’s worse than Lewandowsky’s approach. At least with Lewandowsky the non-sequitur wasn’t so obvious.
Brian H
/ 17/01/2014I believe there’s a conspiracy to promote belief in conspiracies. And another conspiracy to discourage belief in conspiracies. They’re out to get us, and each other!