Gergis et al.

There’s an interesting article in the Conversation by Joëlle Gergis in which she discusses the saga around a paper they initially tried to publish in 2012. The basic story seems to be that there was an error that was only noticed after it had been accepted. They withdrew it, rechecked it thoroughly, went through mutiple rounds of peer review, and have now had it re-accepted.

At the end of the day, however, the result appears little changed and they still conclude – as they did initially – that

observed temperatures in Australasia have been warmer in the past 30 years than every other 30-year period over the entire millennium

The figure below also shows how the new reconstruction (black line) differs from their original result (red line).

Credit: Gergis et al. (2016)

Credit: Gergis et al. (2016)

What makes the story more interesting, is the response of various bloggers to the original paper. It appears (which is not surprising) that they made a mountain out of a molehill. Some of the response were, however, less than pleasant. As Joëlle Gergis says in her article

Viciously attacking a researcher at one of Australia’s leading universities as a “bimbo” and a “brain-dead retard” doesn’t do much to encourage professional climate scientists to engage with the scores of online amateur enthusiasts. Worse still, gender-based attacks may discourage women from engaging in public debate or pursuing careers in male-dominated careers like science at all.

Although climate change deniers are desperate to be taken seriously by the scientific community, it’s extremely difficult to engage with people who do not display the basic principles of common courtesy…

I’m neither female, nor actually a climate scientist, but I can say that engaging publicly in this topic can be remarkably unpleasant. It took me a long time to get use to the kind of vitriol that would be directed at anyone who didn’t dismiss mainstream climate science. If I had any sense, I would have stopped engaging elsewhere, but I think I’ve just come to find it fascinating, and I also know who to ignore and when to not respond. Even so, I’ve ended spending the last few days in a discussion with someone who seems incapable of not responding with a claim that I’m being dishonest.

So, how have bloggers responded to Joëlle Gergis’s article. I’ve seen three so far, all of whom are accusing her of lying. So, a climate scientist points out that being attacked will discourage climate scientists from engaging online, and she is immediately attacked. Kudos. To be clear, I don’t know if what she presents in her article is the whole truth and nothing but the truth, but I don’t really care. Most of this happened before I started engaging in this topic, so I’m only vaguely aware of the background story. From what I’ve seen, however, climate scientists are indeed attacked on blogs that dismiss mainstream climate science, so that bit certainly seems true. It also appears that whatever their error was, it hasn’t really changed the result of their paper, which is kind of the key point.

If people want to go out and do their own analysis and publish their own paper, they should go ahead and do so. Fighting about something that is probably irrelevant and happened a few years ago seems entirely pointless. Okay, it’s pointless if your goal is to gain understanding of climate science. If your goal is to find reasons to critcise climate scientists, then maybe not. Of course, that would then seem to suggest that Joëlle Gergis’s criticism of the conduct on blogs is justified. It does all seem wonderfully ironic to me. There do indeed seem to be some who would like to be taken seriously, but then find excuses why they don’t actually publish anything. They complain bitterly if they feel that they’ve been maligned in some way, but then find reasons why their “attacks” on others are justified. It’s all just a game; I think it might be called ClimateballTM

Advertisements
This entry was posted in Climate change, ClimateBall, Science, The scientific method and tagged , , , , . Bookmark the permalink.

232 Responses to Gergis et al.

  1. If there are people who genuinely think they only object to the science, I would suggest them to start a movement with a new name, to clearly distinguish themselves from the mitigation sceptical movement and to write scientific articles.

  2. Tim Roberts says:

    I don’t personally have much time for Andrew Bolt, but I do agree with the heading of his 2007 Sun Herald article which said “Abuse is the last resort of the fact-challenged”

  3. Tim,
    What I find ironic, is that they will make that very argument while being abusive.

  4. geoffmprice says:

    We’ve all seen enough of these play out to know how credible the blog accusations of “liar!” are likely to be.

    “Although climate change deniers are desperate to be taken seriously by the scientific community, it’s extremely difficult to engage with people who do not display the basic principles of common courtesy…”

    is certainly the resonant quote. The endless circularity is that they take every prior contradiction and dismissal of their (generally baseless) claims as a personal attack, and so believe their consistent abusiveness is *defensive* and a *preemptive response* to the fact that they haven’t been taken seriously by the scientific community and supporters when they (inherently, by some sort of divine right earned via the fact that ‘science is about questioning’) deserve to be.

    The hard, cold fact – around which agreement is probably never likely to come – is that the value of ‘skeptical’ inquiry is judged not by the *existence* of questions but by the *quality* of those questions.

  5. Nick Stokes says:

    “The basic story seems to be that there was an error that was only noticed after it had been accepted.”
    A bit more complicated. They submitted a fairly routine multiproxy analysis for the SH, showing that there had been warming. But in determining significant temperature effect for the original proxies, they said that they had done so after detrending. This is more demanding, and they made a virtue of this, suggesting that detrending was really required. Others don’t do that, and people like Ammann say that detrending isn’t the right thing to do.

    Anyway, it turned out that the numbers they quoted were actually done without detrending, and with detrending the results wouldn’t stand up. This was noted on a blog, and Dr Gergis says they discovered that at about the same time. This was a big problem for the paper as written, but created the situation where they had gathered the data (from the literature) and analysed it (inadvertently) in the conventional way to get a publishable result. Someone else who hadn’t nailed colors to the detrended mast could have published the result. In any case, the valid result was now known, but not published. I was curious to see how that would be resolved.

  6. lorcanbonda says:

    There is no monopoly on trollish behavior. Calls of “bimbo” or “brain-dead retard” are reprehensible — welcome to the modern internet.

  7. Mike Pollard says:

    A storm in a tea cup. Those complaining have little to no knowledge of what is involved in identifying a research question, securing funding, doing the research, preparing manuscripts, revising to address reviewer’s criticisms, publishing, presenting the work to peers and discussing their concerns (both before and after formal publication), and then starting the whole cycle again and again and again. Why anyone would be concerned with the timeline of when the error was found, by who, and the parsing of correspondence between authors and editors when the findings have remained sound is beyond me.

  8. Szilard says:

    IMO, the interesting (not very ClimateBall) questions about the original paper were to do with “Screening Fallacy” issues, how important these might be & whether detrending dealt with them adequately.

    Given that there didn’t seem to be enough proxy data to be useful if they did detrend, back in the day, I assume that the current version doesn’t use it?

  9. Steven Mosher says:

    climateball.

    just one rule

  10. Nick,
    Thanks. I hadn’t realised that they’d made a big deal out of detrending.

  11. If the paper was correct four years ago, they could have just replaced “detrended” by “not-detrended” in either galley proofs or an erratum.

    However, there is a world of difference between data and detrended data.

  12. Dikran Marsupial says:

    Steven “climateball.”

    Scientists should not be forced into playing climateball, it would be better if they were allowed to concentrate on ScienceBall ™ and in a more rational world they would be able to do so, even on contentious subjects such as climate.

  13. Dikran Marsupial says:

    The monumental hubris of some climate “skeptics” never ceases to amaze (rather ironically given the label). A lot of this is down to the depersonalization that comes from discussing peoples work on electronic forums. Oddly enough people tend to be pretty polite to me face to face, in sharp contrast to the reception I sometimes get on blogs. Sadly I think this is a case of “In blogsphere veritas, in rerum sanitas”, in the sense that if you let down the behavioural restraints that apply face-to-face you get to see someones actual inner nature.

    From the Wikipedia page above, the version from the Babylonian Talmud is even more apposite:

    “In three things is a man revealed: in his wine goblet, in his purse, and in his wrath.”

    The second and third elements appear particularly relevant to ClimateBall!

    As to mistakes, we all make them, I certainly do, so we shouldn’t be too hard on those who make mistakes and can acknowledge them and fix them. That should apply to “skeptics” just as it should to “warmists”.

  14. Dikran Marsupial says:

    Richard Tol wrote “If the paper was correct four years ago”

    My ironymeter just vapourised. ;o)

  15. Steven Mosher says:

    “Scientists should not be forced into playing climateball, it would be better if they were allowed to concentrate on ScienceBall ™ and in a more rational world they would be able to do so, even on contentious subjects such as climate.”

    100% agree. nobody should be forced to play climateball.

  16. nobody should be forced to play climateball.

    I think one of the subtleties of ClimateballTM is that you’re playing it whether you like it or not. Even in this case, Joëlle Gergis can completely ignore the critiques of her article, but they still exist and may well end up in the MSM.

  17. Dikran Marsupial says:

    Steven, indeed but that does mean they do have a right to complain (cry) about the nature of the “discussion”, if they only wish to play ScienceBall (don’t cry about the reviews of your papers though ;o). I don’t know if this is the case in this particular instance.

    BTW I think your video suggests the “one rule” should actually be “don’t be an asshole or the moderator/ref/umpire will exclude you from the discussion and will be right to have done so” ;o)

  18. oneuniverse says:

    Gergis in the Conversation :

    Following the early online release of the paper, as the manuscript was being prepared for the journal’s print edition, one of our team spotted a typo in the methods section of the manuscript.

    While the paper said the study had used “detrended” data – temperature data from which the longer-term trends had been removed – the study had in fact used raw data. [..]

    […]

    Yes, we made a mistake – a single word in a 74-page document. We used the word “detrended” instead of “non-detrended”. Atoning for this error involved spending four extra years on the study, while withstanding a withering barrage of brutal criticism.

    This makes it sound as if Gergis et al had intentionally used non-detrended data, but had accidentally typed “detrended” instead in the methods section describing their analysis.

    This is not true . Gergis ea 2012 made a specific argument for using detrended data :

    For predictor selection, both proxy climate and instrumental data were linearly detrended over the 1921–1990 period to avoid inflating the correlation coefficient due to the presence of the global warming signal present in the observed temperature record. Only records that were significantly (p<0.05) correlated with the detrended instrumental target over the 1921–1990 period were selected for analysis.

    Gergis herself acknowledged, in an email to co-authors and colleagues (and separately in an email to the journal), that they (the authors) believed that the above passage correctly described the analysis they thought they’d done.

    [Referring to the passage quoted above] The detrending of records for record selection had been done in another paper on Southern Hemisphere temperate variations that we had been writing simultaneously, so we wrongly assumed the same thing had been done in the Australasian paper.

    So by her own admission then, it wasn’t a typo (which is when one means one thing, but writes another by mistake). She now says it was. Which version of her story to believe? In my view the evidence strongly supports the case that it wasn’t a typo.

  19. oneuniverse,

    This makes it sound as if Gergis et al had intentionally used non-detrended data, but had accidentally typed “detrended” instead in the methods section describing their analysis.

    I must admit that I had interpreted this the other way around; that they had intended to detrend, but had typed the wrong term in the paper.

    However, my overall point is that I don’t hugely care. Partly because I don’t see what we achieve by running around shouting liar, liar. Partly because I can’t see how it would ever really be resolved; one side will say it’s not true, the other will say it is; it’ll just be another example of an issue that divides people. Partly, because I do have sympathy with those who’ve been abused and insulted on climate “skeptic” blogs. Scientists do make mistakes; it happens. Ultimately our goal should be to improve our understanding of the system being studied. Fights about whether or not something someone said in the media, or on a blog, is true, or not, is unlikely to be constructive.

    Maybe I can ask you a question. What would be achieved by trying to determine who is lying and who is not, or by delving into something that mainly happened 3 to 4 years ago? I can’t see how this would be a positive, or constructive, thing to do. Furthermore, a lot of Joëlle Gergis’s article was about the abuse she recieved as a consequence of the error in this paper. How does maligning her further help to reduce this issue, which I think is real.

  20. oneuniverse says:

    Actually, Gergis’ account is a bit more contradictory than I thought. She also writes in the article:

    When we checked the computer code, the DETREND command said “FALSE” when it should have said “TRUE”.

    But as noted, she additionaly states :

    Yes, we made a mistake – a single word in a 74-page document. We used the word “detrended” instead of “non-detrended”.

    So according to Gergis, there were two typos :

    – The code should have had DETREND set to TRUE instead of FALSE, so it should have detrended the data.
    – The methods sections should have said “non-detrended” instead of “detrended”.

    If one makes both of these (contradictory) corrections, there’s still a mismatch between the description and the actual analysis !

  21. Oneuniverse,
    Yes, that’s why I interpreted her article as saying they had intended to detrend, but had typed that in the paper, and then discovered that the flag was set to FALSE.

    Maybe you can have a go at answering my question, because my main point is that I don’t really see the point of actually debating this in great detail as I can’t see what that would achieve.

  22. Dikran Marsupial says:

    “So according to Gergis, there were two typos : ”

    no, it doesn’t you appear to be double counting. If the intention was that the flag should have been TRUE, then what was originally written in the paper was a correct statement of what they thought they had done.

    “If one makes both of these (contradictory) corrections”

    Why would they do that? There was one correction required, namely to set the flag to TRUE and repeat the analsysis, which AFAICS was what was done.

    I think you are trying a bit too hard to find fault here.

  23. Joshua says:

    ==> Partly because I don’t see what we achieve by running around shouting liar, liar. =>>

    Obviously, the advantage is that you can use such a circumstance to characterize climate scientists as a group, and the field of climate science in general.

    When the “motivation” exists, extrapolating from examples, even if their representativeness hasn’t been determined, is the proximal goal.

  24. Obviously, the advantage is that you can use such a circumstance to characterize climate scientists as a group, and the field of climate science in general.

    Well, yes, that’s the obvious one. I was wondering if Oneuniverse could come up with something more constructive than that.

  25. oneuniverse says:

    Dikran Marsupial:

    no, it doesn’t you appear to be double counting. If the intention was that the flag should have been TRUE, then what was originally written in the paper was a correct statement of what they thought they had done.

    If the intention was to detrend, why did Gergis also write :

    Yes, we made a mistake – a single word in a 74-page document. We used the word “detrended” instead of “non-detrended”.

    It’s contradictory. She’s saying the description of methods should have said “non-detrended”, while the code should actually have detrended.

    ATTP:

    However, my overall point is that I don’t hugely care. Partly because I don’t see what we achieve by running around shouting liar, liar.

    I haven’t called Gergis a liar, or made any kind of personal remark about her. I’m pointing out, in a reasoned way I hope, what seem to be obvious inaccuracies and inconsistencies in her recent article referenced in your blog post. I’m surprised no-one else has pointed them out.

    Partly because I can’t see how it would ever really be resolved; one side will say it’s not true, the other will say it is; it’ll just be another example of an issue that divides people.

    If you read the FOIA’d correspondence, which just takes a few minutes, it’s actually quite clear (IMO) – it’s not a matter of “he said/she said”, except for the question of whether the authors spotted the detrending issue independently of Jean S and others at CA. The authors originally meant to do the analysis with detrended data, described it as such in the paper, realised afterwards that they’d done it with non-detrended data (both methods can be argued for), and had a discussion as to how to proceed for resubmission (mainly, whether to alter the manuscript so that it correctly described what they actually did ie. use non-detrended data, or alter the method to correctly follow what the manuscript says they did, or do both analyses and compare and discuss their merits. Hopefully they ended up doing the latter.)

    Maybe I can ask you a question. What would be achieved by trying to determine who is lying and who is not, or by delving into something that mainly happened 3 to 4 years ago?

    Gergis is the one who published the recent article making the ‘one-word typo’ claim, which can easily be shown to be incorrect, as far as I can tell. You’ll have to ask her why she did this. I find it baffling.

    Furthermore, a lot of Joëlle Gergis’s article was about the abuse she recieved as a consequence of the error in this paper. How does maligning her further help to reduce this issue, which I think is real.

    I condemn any and all abuse she’s received, as I assume does everyone here. Taking issue with what she wrote re: one of the the main points referenced in the headline of her article, the one-word typo, however, is not maligning her (if that’s what you meant).

  26. Oneuniverse,
    I realise you haven’t called her a liar, but many others have.

    Taking issue with what she wrote re: one of the the main points referenced in the headline of her article, the one-word typo, however, is not maligning her (if that’s what you meant).

    No, I don’t think that taking issue with what she has said is maligning her, but you still haven’t explained why delving into this is something worth doing. What would be achieved?

    Of course, someone could write a response with their own views, and that would be fine. I’m just not sure what would be achieved by discussing in detail whether it was simply a typo, or a mistake that she should have acknowledged in some different way.

  27. Dikran Marsupial says:

    one universe wrote “If the intention was to detrend, why did Gergis also write :

    Yes, we made a mistake – a single word in a 74-page document. We used the word “detrended” instead of “non-detrended”.”

    Because that would make the paper state what was actually done rather than what they thought they had done. There are not two typos, just two ways of fixing one error, either you fix the code and redo the experiment the way you intended OR correct the paper so that it accurately documents what you actually did. In the end they did the former. As I said, you appear to be trying to hard to find fault.

  28. Dikran Marsupial says:

    oneuniverse, what is your opinion of the science in the revised paper?

  29. izen says:

    @-Dikran Marsupial
    “oneuniverse, what is your opinion of the science in the revised paper?”

    Any opinion about the science in the paper is irrelevant.
    The issue is that this can appear to be climate scientists doing ‘tricks’ with the data, to get the answer desired by the secret NWO which want to tax us more.

  30. Magma says:

    At the risk of straying off-topic, compare the painstaking work and corrections of Gergis and her coauthors to some recent claims by solar physicist Valentina Zharkova, whose forecast of an upcoming deep solar minimum was widely quoted by our skeptical friends last year. Some of her comments on the potential global cooling effects that this could induce could have been partly excused by her having been caught off-guard on an unfamiliar topic. She subsequently engaged in a confused muddle of both refuting and repeating her statements.

    However Zharkova is now back in full force.

    Prof Zharkova said: “We have now established a mathematical law which others can use to apply to this area of research, and so far we have been able to match our research with proven meteorological records dating back 3000 years to 1000 BC.

    “This has given us the confidence to predict what will happen to solar activity in the future decades. This decrease poses a question about expected reduction of the temperature of the planet in the coming years because the sun, as we are confidently predicting, will enter into a grand minimum beginning in 2020 – the first such one since the Maunder Minimum.

    “We confidently predict this minimum will last for three cycles (33 years), not as long as the last one, but during this time global temperature may fall by an average of 1.5°C although there will be fluctuations across the globe.”

    http://www.chroniclelive.co.uk/business/business-news/mini-ice-age-could-freeze-11607587

    I’m puzzled by some of these Russian astronomers, including Zharkova, Helen Popova (“There is no strong evidence that global warming is caused by human activity”), and Habibullo Abdussamatov, and their strange forays into climate science. Not knowing much about the field or the state of Russian institutes, is this just a small group of unrepresentative scientists, like judging Harvard by Willie Soon’s peripheral association would be?

  31. Yes, I saw that. Rather odd that Zharkova appears to now be an expert at climate, given that last year she seemed slightly confused as to why you needed to divide the change in TSI by 4 to get the change in Solar forcing.

  32. Willard says:

    A blast from Judy’s past:

    Thank you, Professor Curry, for your work.

    Please cite the 2015 paper by Professor Valentina Zharkova et al on Sun’s weakening magnetic field in solar cycles 21-23,

    http://www.nature.com/articles/srep15689

    and news reports of an imminent “Little Ice Age,” “Ethics Crisis” and “Irregular heartbeat of the Sun driven by double dynamo.”

  33. Eli Rabett says:

    Eli Rabett’s rule of the random Russian holds.

  34. Dikran Marsupial says:

    izen wrote “Any opinion about the science in the paper is irrelevant.
    The issue is that this can appear to be climate scientists doing ‘tricks’ with the data, to get the answer desired by the secret NWO which want to tax us more.”

    alternatively the fuss about whether the climate scientists is doing “tricks”* with the data is just an excuse to ignore what the science actually says, which may have relevance to policy. If oneuniverse accepts the findings of the revised paper that makes the nature of his/her objection more clear. IMHO the only thing that matters is whether the science is correct/robust as the correct course of (in)action depends on the science not the intentions of the climatologists or the convictions of the politicians etc.

    * “trick” in a mathematical context just means some neat/clever way of working something out, it doesn’t imply any nefarious intent.

  35. Dikran Marsupial says:

    Magma wrote “She subsequently engaged in a confused muddle of both refuting and repeating her statements.”

    she is a normal human being then? ;o)

  36. Willard says:

  37. oneuniverse says:

    Dikran Marsupial:

    There are not two typos, just two ways of fixing one error, either you fix the code and redo the experiment the way you intended OR correct the paper so that it accurately documents what you actually did.

    In either of those cases, Gergis is wrong to characterise the fix as just a single word that needs to be corrected. Either they have to alter the manuscript to remove the part describing and justifiying the detrending and add a justification for not detrending (if they decide to go with non-detrended data), or they have to redo the analysis using detrended data. According to Karoly’s correspondence in 2012, if they redid their analysis for proxy selection using detrended data, there would have been “only about 9 selected proxies and only one prior to 1400. No reliable reconstruction prior to 1400.” Assuming that Karoly was correct, their results would have been materially different, and again required changing significantly more than just a single word.

    re: the new paper
    I haven’t found an unpaywalled version yet, so haven’t had a chance to read it.

  38. Dikran Marsupial says:

    oneuniverse “In either of those cases, Gergis is wrong to characterise the fix as just a single word that needs to be corrected.”

    I hope that when people criticize your work they are not so mercilessly pedantic. Human beings don’t always communicate with perfect clarity and precision. You are making a mountain out of a molehill, and inaccurately at that. If the paper has a section that explains the need for detrending, but the correctly states what was actually done, then that is satisfactory (as it basically points out a flaw/shortcoming in what they had done). AFAICS fixing that one word does make the paper technically correct, if not as good as it could have been, which is presumably why they didn’t take that approach.

    There is a thing called Hanlon’s razor which is useful in such circumstances. I normally adopt a less stridently worded version, such as “always attribute to others the most favourable motivation that is consistent with the observations”. This is not rocket science, just an application of the “golden rule” and also helps to implement a degree of self-skepticism in yourself.

  39. Willard says:

    > it would be better if they were allowed to concentrate on ScienceBall ™ and in a more rational world they would be able to do so, even on contentious subjects such as climate.

    I agree: unverifiable stories, Gremlins, intentions, personal correspondence, and concerns about tone all belong to ClimateBall ™.

  40. Nick Stokes says:

    1U,

    Either they have to alter the manuscript to remove the part describing and justifiying the detrending and add a justification for not detrending (if they decide to go with non-detrended data), or they have to redo the analysis using detrended data.

    The original blog result was that it doesn’t work with detrended data, as Karoly also said. As you quoted, their argument was

    For predictor selection, both proxy climate and instrumental data were linearly detrended over the 1921–1990 period to avoid inflating the correlation coefficient due to the presence of the global warming signal present in the observed temperature record.

    Others in this situation do not detrend. Their argument is that the resulting correlation coefficient is not “inflated”; agreement of the signals in trend is legitimately part of the concordance which says that the proxy is responding to temperature (and an important part). That is why I said above that the perverse situation was that they have a result that was correct and publishable according to normal conventions, but were stranded by their incorrect claim that detrending was required. They had to find some graceful way of disavowing that claim.

  41. Willard says:

    > They had to find some graceful way of disavowing that claim.

    Gremlins may have been less suboptimal. There are many in the auditing repertoire. My favorite is simply a new blog post.

  42. Dikran Marsupial says: “I hope that when people criticize your work they are not so mercilessly pedantic.

    +1

    I had perfectly understood from the full context that the flag was wrong.

    I really hope that oneuniverse does not expect me to invest time to read his work when he operates like this.

  43. Magma says:

    @ Willard: Did the Auditor ever turn his penetrating gaze to the work of T‍o‍l, Lomborg or Spencer & Christy? Or would that have been unsporting, like shooting fish in a barrel?

  44. Dikran Marsupial says:

    Magma, it was directed at the (obviously incorrect) Douglass et al (which included Christy). My contributions were later described as “arid pontificating” (despite it being a pretty straightforward thought experiment demonstrating the error), However Steve was pretty unwilling to discuss it, while missing the point completely. He did however “turn his penetrating gaze” towards that particular barrel.

  45. Willard says:

    No idea, Magma. I don’t think he ever criticized Richie on his blog. For everywhere else, that’s a biographical issue – turn it over to historians.

    Which leads us to one way to reduce the scope of ClimateBall ™:

    [Shovelling] This is an X matter, turn it over to X specialists.

    A not too distant usage, in which Gergis gets a cameo.

  46. Willard says:

    Oh, and this:

    God I’ll miss PK Subban.

  47. oneuniverse says:

    Nick, I thought both methods can be argued for, so is “incorrect” maybe a bit strong? Gergis writes in the Conversation : “Both raw and detrended data have been used in similar studies, and both are scientifically justifiable approaches.”. Karoly has made similar comments. Or has that debate been resolved ? Otherwise, I think your first comment here sums up the situation with the paper well. (Much better than Gergis’ article).

    Dikran Marsupial, I’m not sure why you’re telling me about Hanlon’s Razor, as I’ve made no comments or assumptions about Gergis’ intentions or competence, positive or negative.

    AFAICS fixing that one word does make the paper technically correct, if not as good as it could have been, which is presumably why they didn’t take that approach.

    As far as I can see, there’s no one-word change one can make to the paper that would’ve made it technically correct. Changing “detrended” to “non-detrended” would lead to farcical and untrue statements such as “For predictor selection, both proxy climate and instrumental data were linearly non-detrended over the 1921–1990 period to avoid inflating the correlation coefficient due to the presence of the global warming signal present in the observed temperature record.” Alternatively, if the one-word change were FALSE to TRUE for the DETREND command (what Gergis puzzlingly describes in the article as “taking the easy way out”), that would’ve meant that the reported results would knowingly have been for a different analysis than that reported, which would be scientific fraud.

    Just so I can make sure I’m on the same page, would you mind describing precisely which word, in which sentence or sentences, you think can be changed in the manuscript that would make it technically correct?

  48. oneuniverse says:

    Actually, on re-reading I think Gergis is referring to changing the word “detrended” (and not the FALSE flag) when she wrote “Instead of taking the easy way out and just correcting the single word in the page proof, we asked the publisher to put our paper on hold “, my mistake. But in that case, as I wrote, that would lead to farcical and untrue statements in the paper.

  49. BBD says:

    And years later, after all the auditing, out pops a hockey stick.

    All that fuss for nothing.

  50. BBD says:

    Might as well stick in the pretty picture:

  51. Steven Mosher says:

    Thanks for getting the point dikran.

    Here is another view.

  52. Steven Mosher says:

    “I think one of the subtleties of ClimateballTM is that you’re playing it whether you like it or not. Even in this case, Joëlle Gergis can completely ignore the critiques of her article, but they still exist and may well end up in the MSM.”

    Yes I often comment to Robert rohde that he is playing climate ball when people criticize his work on blogs and he doesn’t even read it. For Climate ball to be meaningful as a term it would seem that their must be a way to not play it.
    Does feynman play climate ball?

    Anyway you look at it blogs and the Internet is a moshpit.grad students should get training on when to jump in, when to call in an enforcer, when to avoid it and styles of engagement should you choose to engage directly. Personal I would tell folks to study zeke style and Robert way style and they will do just fine. And they need a pet bulldog. The more dispensable
    The better.

  53. Harry Twinotter says:

    The Conversation got out the knife and removed many of the comments made on that article (including some of mine). A bit over the top, but probably necessary in this case as the article was becoming a magnetic to climate change deniers who lose their minds over any study that shows a hockey stick curve, and want to discredit it.

    My take on it is the use of “raw” vs detrended data made little difference to the result. But considering the public interest in the original study they decided to be thorough and check that this was true, instead of just changing a word in the methods section to match the analysis program.

  54. Ken Fabian says:

    I think there is a desire by those rejecting mainstream climate science (or the responsibility for emissions that acceptance of it brings) for incidents like this to become a default excuse to begin with a presumption that the revised paper – and all climate science papers – are error filled. And/or that the peer review process is fundamentally, rather than occaisionally, letting us down. But, then the loudest critics in this case have already decided that widespread belief that climate science papers are error filled and the peer review process is flawed is something to be promoted wherever and however possible.

    It’s one thing for someone having both the competence and real intention to thoroughly review peer reviewed papers to (temporarily) withhold judgment, but for those who lack the expertise and/or have no real intention to do such a review, it is a blanket rejection of both the practices and the conclusions of scientific inquiry.

  55. Willard says:

    > For Climate ball to be meaningful as a term it would seem that their must be a way to not play it.

    Playing the ball ought to be enough. Safer is also to never play the man. More generally:

    Once upon a time, before the Little Ice Age, an old Zen punk solved the hardest problem of the Internet.

  56. Dikran Marsupial says:

    oneuniverse “Dikran Marsupial, I’m not sure why you’re telling me about Hanlon’s Razor, as I’ve made no comments or assumptions about Gergis’ intentions or competence, positive or negative.”

    The point was that you are looking for inconsistencies to such an extent that you are finding more than is actually there. Hence view others in as favourable light as is consistent with the observations (which is essentially what Hanlon’s razor tells us to do).

    “Just so I can make sure I’m on the same page, would you mind describing precisely which word, in which sentence or sentences, you think can be changed in the manuscript that would make it technically correct”

    I think this makes my point about merciless pedantry. The point is that either you change the paper so that it accurately reflects what was actually done (even if it would have been suboptimal in the authors opinion) or you redo the experiment. It doesn’t really matter if it is just one word that needs to be change or a few paragraphs, the point Gergis is making is that the real error was that the paper as written did not accurately reflect what they did. So the Conversation article contained a slight exaggeration/hypebole, so what? It isn’t as if it is a journal paper, its a blog!

    I think you are making a mountain out of a molehill, especially as the “change one word” wasn’t even the option they actually took!

  57. Dikran Marsupial says:

    Steven Mosher wrote “Thanks for getting the point dikran.”

    I actually understood something via a baseball analogy? I hope nobody from my cricket club ever finds out! ;o)

  58. Dikran Marsupial says:

    BTW interesting video, it is nice to see that it is automatically acceptable to be informal when speaking to yourself! The bit at about 6:39 is very relevant, we should always assume that the person we are speaking to on blogs are “older” than us if we want the discussion to be productive.

  59. jibalt says:

    “I’m surprised no-one else has pointed them out.”

    They haven’t pointed them out because they aren’t there. What people HAVE pointed out is that your interpretations are incorrect.

  60. oneuniverse says:

    jibalt, actually they haven’t. My first comment is correct, AFAICT. There was no one-word fix that would have made the manuscript technically correct, and Gergis’ claim to the contrary is untrue. Dikran Marsupial appears to concede this point by saying that the article “contained a slight exaggeration/hypebole”. It was rather more than that. Gergis wrote: “Instead of taking the easy way out and just correcting the single word in the page proof, we asked the publisher to put our paper on hold and remove the online version while we assessed the influence that the different method had on the results.” Apart from the fact that there’s no one-word “fix” available, switching from using detrended data to non-detrended data would’ve changed their method of analysis. It seems highly unlikely that they would’ve been able to update the manuscript to change the description of their method, with accompanying changes to the reasoning, and the literature cited, without being required to resubmit it for review. (If the authors can make changes to their described methodology after review, the work is not properly peer-reviewed as the phrase is generally understood).

    By the way, if you want to show that I’m wrong about the ‘one-word’ claim, you could just answer the question I asked Dikran Marsupial, which he declined to answer (and instead rather disappointingly accused me of “merciless pedantry” – no Dikran, I was just asking you provide the specifics for your claim that “AFAICS fixing that one word does make the paper technically correct [..]”). My question was : would you mind describing precisely which word, in which sentence or sentences, you think can be changed in the manuscript that would make it technically correct? The manuscript is here.

  61. Dikran Marsupial says:

    oneuniverse “I was just asking you provide the specifics for your claim that “AFAICS fixing that one word does make the paper technically correct [..]”)”

    It pretty much does. Technically correct means that the paper describes what was actually done and gives valid interpretation of the results. It doesn’t mean that it was done optimally. You are being tiresome, so the blog article had a little hyperbole, so perhaps I was a little imprecise. The point is that the paper could have been fixed in two ways, either it could have been trivially changed (so maybe not just one word, but still trivially) to document what was actually done, or the study could have been repeated correctly and the paper rewritten. I’m afraid that it is merciless pedantry to focus on whether it is a single word change or not while ignoring the substantive point (that Gergis took the more onerous path in order to write as good a paper as they could rather than take the easier way out).

    ” (If the authors can make changes to their described methodology after review, the work is not properly peer-reviewed as the phrase is generally understood)”

    This is a comment is just silly. Firstly errors do get through peer-review all the time. Secondly if the author is made aware of such an error then the correct thing to do is to contact the editor and explain the situation. Withdrawing the paper and resubmitting a corrected one for a further round of peer review is the common sense solution.

    “Dikran Marsupial appears to concede this point by saying that the article “contained a slight exaggeration/hypebole”. ”

    The fact that I had to point this out to you is evidence of your uncharitable pedantry.

  62. oneuniverse says:

    It’s not pedantry or uncharitable to point out that a claim made in the lede section of the article (and echoed in the headline) is misleading and false. There was no one-word alteration that could’ve simply been made to the proof of the manuscript. This was a simple observation I made in my first comment, it remains true.

    Look, Gergis wrote : “one of our team spotted a typo in the methods section of the manuscript”. This is simply untrue. The methods section did not contain a typo – it accurately described what they intended to do, and what they thought they’d done, as admitted by Gergis herself in correspondence at the time.

    The actual “typo” (or possibly oversight in reusing code from elsewhere) was accidentally setting the DETREND flag to FALSE instead of TRUE in the code (not the manuscript). Fixing that would have required redoing the calculations, which it seems would’ve led to materially different results, according to co-author Karoly (“only about 9 selected proxies and only one prior to 1400. No reliable reconstruction prior to 1400″ ). Whatever the exact nature of the new results, J Climate wanted the submission of the paper withdrawn if they had to redo the analysis with different methodology (JClim editor John Chiang: “Also, since it appears that you will have to redo the entire analysis (and which may result in different conclusions), I will also be requesting that you withdraw the paper from consideration. “)

    It’ll be interesting to see how the new paper arrives at the conclusion that “the differences between using detrended and raw correlations to screen the predictor networks, as well as between using field mean and local correlations, are minor “, since this is significantly different to what Karoly found at the time. Ideally I’d like to examine the code to see what was actually done (once bitten, twice shy – or is that uncharitable).

  63. Dikran Marsupial says:

    “It’s not pedantry or uncharitable to point out that a claim made in the lede section of the article (and echoed in the headline) is misleading”

    It’s only misleading if you cant recognize a piece of mild hyperbole. Making a fuss about it is unreasonable, uncharitable and pedantic.

    “The methods section did not contain a typo – it accurately described what they intended to do,”

    you are still missing the point, there are two ways of characterising the problem: (1) the paper not accurately reflecting what they actually did (2) not actually performing the experimental work they intended. It doesn’t really matter which way round you look at it, so it isn’t really a typo if you are being pedantic about it, but someone with an ounce of common sense and willing to take a charitable view will look for the most reasonable interpretation, rather than making a fuss about the wording of a blog article, while ignoring the science almost completely AFAICS.

    Please let me know when you have something to say about the science, rather than pedantry about the wording of a blog article, I’ll leave it at that until then.

  64. Dikran

    I think one of the underlying issues here is
    people ( like Jean S) find an error in a paper

    Then
    A) one side tends to get hyperbolic about the error.. all science is broken.. crisis in science
    B) one side tends to call all such things “typos”
    C) and then they refuse to give credit for finding what they argue is a trivial error.

    I don’t think litigating this gets us anywhere, been fighting this kind of stuff too long and the
    issues of ‘priority’ ( who found X first ) and the severity of problems ( are results remain largely intact) are always going to be contentious in any science ( cause priority is really important ).
    So take those already contentious and hard to resolve issues and through some climate war gas on the fire and you get what we have today.

    still recall ravetz comment to me. “The problem is Steve we dont have a theory of making mistakes”

  65. Dikran Marsupial says:

    Yes A-C is definitely human nature in action. Trying to point out errors without making someone defensive can be difficult, especially where people are already looking to take offense and even more so on electronic forums which lack the non-verbal cues that prevent misunderstanding face to face.

    “theory of making mistakes” :o)

  66. oneuniverse says:

    It’s only misleading if you cant recognize a piece of mild hyperbole.

    Certainly, if you know her claim is false, you won’t be misled.

    However, if a scientist is discussing her paper and makes a specific claim like “Yes, we made a mistake – a single word in a 74-page document. We used the word “detrended” instead of “non-detrended” ” – I think most people unfamiliar with the details will take her at her word, rather than assume that she’s exaggerating, especially when the claim is reinforced in the same article by repeated other mentions of an apparent single word error, without any hint (that I can see) that this is in some way meant to be interpreted as hyperbole :

    – “How a single word sparked a four-year saga of climate fact-checking and blog backlash”
    – “one of our team spotted a typo in the methods section”
    – “It turned out that someone else had spotted the typo too”
    – “Instead of taking the easy way out and just correcting the single word in the page proof”

    By the way, taking a few moments to write a comment pointing out what I see as a misleading error in the article, and responding to the replies, doesn’t mean that I’m “ignoring the science” – please.

  67. oneuniverse says:

    Harry Twinotter:

    My take on it is the use of “raw” vs detrended data made little difference to the result.

    According to Table S1.2 and Fig. S1.3 in the supplementary material to Gergis ea 2016, using detrended data and correlating the proxies directly to the target field mean (which I think is the method used in Gergis 2012) selects only 9 proxies, and yields a reconstruction going back only to about 1600. The reconstruction used in the main text, going back to 1000, also uses detrended data, but appears to rely on a more relaxed selection method : if a proxy significantly correlates with any of the grid-cell temperatures within a 500 km radius, it passes the text. This method does seem to be mostly insensitive to the raw/detrended choice (for these datasets).

  68. Willard says:

    oneuniverse,

    You’ve said your thing more than once now. Parsing what is or is not a hyperbole, under what conditions a statement can be misleading, and whatnot, should stay at Lucia’s. As far as I am concerned, my eyes ordinarily tend to glaze over such dispute. Unless I am being challenged to play parsomatics, am committed to play a dictionary game, or unless it has philosophical implications that would interest me. None of these apply to this line of argument, which is not the topic AT introduced anyway.

    Considering the billions of ClimateBall ™ episodes over the words “unprecedented” and “delete,” I’m not even sure your first example of hyperbole is really one. I’d rather use your “taking a few moments to write a comment pointing out” as a good example of the opposite of a hyperbole.

  69. OT alert (dammit, again):

    Willard, I now have my copy of Brad Warner’s Don’t Be a Jerk. I was expecting something fun and, well, punkish. Instead I got a book that is a candidate for if I could only have one book, this might be it. I’m serious, thank you, from the bottom of my confused, mystical atheist being.

  70. Steven Mosher says:

    ““theory of making mistakes” :o)”

    Then the conversation got interesting.

    Look at the mistakes here

    Or how do tell, on its face, the difference between a mistake and innovation

  71. Dikran Marsupial says:

    it’s not a bug, it’s a feature?

  72. It can sometimes depend on whether your objective is to reflect reality or some distorted version of it.

  73. BBD says:

    It’s still a hockey stick, despite all the years of contrarian fussing. And that’s what counts in the end.

  74. AndyL says:

    [What happens at the Auditor’s should stay at the Auditor’s. Thank you for your concerns. -W]

  75. AndyL says:

    [Mod – sorry, I’m largely on a break with a bit of free time and would rather not moderate a contentious comment thread on my blog, especially as I may not have un-interupted internet access. If you post this on CA I’ll endeavour to respond there.]

  76. Steven Mosher says:

    “It’s still a hockey stick, despite all the years of contrarian fussing. And that’s what counts in the end.”

    Ah no.

    The problem is it authorizes the use of several suspect practices.

    A) comparing a proxy to multiple temperature fields to find a correlation WITHOUT correcting
    for this in uncertainties. Its pretty bad with 5 degree bins out to 500km, it would be hilarious
    with 1 degree bins out to 500km.
    B) using “leads” Physically that means the proxy predicts temperature. Unphysical is bad.

    c) screwing up your calibration and verification time periods. Basic stuff.

    That said, boneheaded mistakes in Paleo, don’t change the physics.

  77. BBD says:

    Ah no.

    Ah yes. Redone, and still a hockey stick, like all the others. Show me something that results in a significant effect on the results.

    As I have said many times, contrarian discourse is basically making a fuss over nothing in order to create the false impression that there are ‘problems’ with ‘the science’. Really, there aren’t.

  78. Steven Mosher says:

    “Ah yes. Redone, and still a hockey stick, like all the others. Show me something that results in a significant effect on the results.”

    Ah no this is NOT red one.

    This is basic.

    Lets suppose you have 10 time series

    Time Series A1-A10: The population of immigrants in 10 different areas of london over time
    Time Series B: The time series of Rape reports in one of those districts.. say B5

    Now you want test if B is correlated to a the population of immigrants.

    Suppose I started by testing the correlation between B and all of A1-10 averaged

    And I found nothing..

    So Then I tested A5… and found nothing

    So they I tested A1-4, and A6-10.. and found that yes A8 was correlated

    One can do this, but you have to account for the multiple tests. They didnt. That is a significant effect on the results.

    Suppose now that I added lags and leads to my analysis.. again I am fishing for a correlation.
    That might be OK in EDA.. but they are not merely doing EDA

    Here is what I predict. I predict you wont get Dikran to come on here and endorse this methodology as good practice.

    Further if you have a calibration period of 1920 – 1980, I will predict that Dikran will say
    your verification period should not overlap with that. For example you would not choose
    1910-1930 as a verification period. Ask him if he would do that or endorse that.

    As to results. You dont have results from doing it correctly. So speculation about the answer is just denier crap from our side.

  79. Gentlemen,

    Your engagement on these issues is more than welcome, but please avoid pointless food fight. There’s Judy’s open threads for that.

  80. BBD says:

    One can do this, but you have to account for the multiple tests. They didnt. That is a significant effect on the results.

    Here’s what I predict: SM will not publish a reply demonstrating that:

    You dont have results from doing it correctly.

    I would actually be pleased to be mistaken in my prediction as I fully endorse sound science (TM). As I’m sure do most present here.

  81. Dikran Marsupial says:

    Steven Mosher wrote: “The problem is it authorizes the use of several suspect practices.”

    Publishing something in a peer-reviewed paper doesn’t authorize anything, for that to be true would require a peer-review system that was designed to prevent any errors appearing in print, rather than as a basic sanity check (which is all that peer-review can be expected to be, given the resources that can reasonably be assigned to it).

    In any case, scientific methods are constantly being refined, so not-so-good methods (that are typically used in early studies) are gradually replaced by better ones, so any “authorization” would only be tentative and of limited time-span.

    People use suspect methods in papers all the time (e.g. much of NHSTing), it isn’t really “authorized” in any sense, just the course of least resistance.

  82. mwgrant says:

    Steven Mosher

    One can do this, but you have to account for the multiple tests. They didnt. That is a significant effect on the results.

    Mosh, clarification/curiosity Q–are you specifically getting at the loss of statistical power here? and/or degree of independence (or not) of the data set vis-a-vis the series’ locations.

  83. mwgrant says:

    typo– “…independence (or not) of the data sets

  84. With a PR element to his professional work, ATTP should perhaps reflect on how this kerfuffle might be perceived by lay readers like myself who are genuinely interested in climate science but absent technical competence, rely heavily on behavioural indicators to assess credibility – much like a jury member does. In the conversation piece, Gergis minimises what was in effect, a gross error (albeit honestly made) and then attempts to play the victim by deflecting criticism, accusing the dark forces of denial of harassment. Her deliberately vague, loosely written insinuations ensuring that the muck is spread as widely as possible. To be clear – the article is political spin.

    For those who have read, or are prepared to read more, she has lost credibility as an ‘expert witness’. I’m afraid those here supporting her version of events (or trying to sweep it under the carpet), don’t seem to realise that in doing so, it undermines their own credibility. I’m left wondering that perhaps they don’t care? Or maybe people who are prepared to ‘read more’ simply aren’t the audience. After all, it’s much easier to convince those who don’t hold an opinion than convert those who do.

    She could have said; “We messed up first time around. Simple error but we’ve fixed it now and produced a better paper” and stopped there. By introducing the backstory in the conversation article, she has re-opened the wound and her highly dubious version of history deserves scrutiny by those she accuses.

    Unfortunately for Gergis, instead of simply accepting her revised paper as honest science, it’s now all too easy for me to think that the original paper was indeed fundamentally flawed and that it has taken 4 years to figure out a way to get a similar result in a desperate attempt to save face.

  85. war,

    With a PR element to his professional work, ATTP should perhaps reflect on how this kerfuffle might be perceived by lay readers like myself who are genuinely interested in climate science but absent technical competence, rely heavily on behavioural indicators to assess credibility – much like a jury member does.

    If by PR you mean Public Relations, then this is not true. I’ve never had a PR element to my professional work. If you want to judge someone, and their work, on the basis of their behaviour, that’s of course your choice. However, at the end of the day, scientists are human and not all behave as ideally as we make like. Also, there is a difference between judging individuals and judging an entire discipline. Given that what is meant to matter is the quality of the research itself, not the behaviour of the scientists involved, it is quite rare to see an individual’s behaviour being called out by others in the field. There are exceptions, of course, but I don’t think that in general it is fair to judge an entire discipline on the basis of the public behaviour of a few.

    I’m afraid those here supporting her version of events (or trying to sweep it under the carpet), don’t seem to realise that in doing so, it undermines their own credibility. I’m left wondering that perhaps they don’t care?

    Like me, perhaps they’re reluctant to be overly critical of a climate scientist, given how they are often treated on blogs.

    By introducing the backstory in the conversation article, she has re-opened the wound and her highly dubious version of history deserves scrutiny by those she accuses.

    Some of it seems quite plausible. Climate scientists are regularly maligned on “skeptic” blogs. Also, what would further scrutiny actually achieve? As far as I can tell, it would simply polarise things further.

  86. Dikran Marsupial says:

    “lay readers like myself who are genuinely interested in climate science but absent technical competence, rely heavily on behavioural indicators to assess credibility – much like a jury member does.

    Yikes! Personally I would rather have a jury of people with better control of their cognitive biases and judged the case based on the evidence. I worry about society sometimes…

  87. Dikran Marsupial says:

    BTW I recommend people analyzing other peoples accounts of events should watch Rashomon, and consider whether they share some of the issues of the woodcutter of the story (who may have viewed himself as an independent observer of the events). Who’d be a judge? ;o)

    Yes of course there is likely to be an element of “Rashomon” in Gergis’ account, most of us understand human nature well enough to take that as read and account for that in our evaluation, but I think there is undoubtedly a bit of “woodcutter” in warofthewolds’ account as well.

  88. Dikran Marsupial says:

    It is also interesting to consider what “behavioural indicators” warofthewolds provides in his/her comment, for instance this uncharitable extrapolation from the facts:

    “it’s now all too easy for me to think that the original paper was indeed fundamentally flawed and that it has taken 4 years to figure out a way to get a similar result in a desperate attempt to save face.”

  89. ATTP “If by PR you mean Public Relations, then this is not true. I’ve never had a PR element to my professional work”
    In which case I am misinformed and offer my unreserved apologies on that aspect of my comment.

    DM: What do you call a man in control of his cognitive biases? A. deluded
    I mention the jury analogy because as we all know, expert witnesses regularly offer contradictory evidence in court and for the judge/jury, credibility is the marginal difference. Gergis has been somewhat economical with the truth in her article, but she didn’t need to be. Although illogical to extrapolate that tendency to everything she said, she has unnecessarily damaged her credibility. People often forgive but rarely forget these character lapses.

    DM: “judged the case based on the evidence” well that would be a novel approach.

    DM: “.. uncharitable extrapolation from the facts:”
    Yes it is an uncharitable extrapolation (nice phrase) and was intended to be so for effect. ‘It’s all too easy’ is of course, the point.

  90. chris says:

    Although illogical to extrapolate that tendency to everything she said, she has unnecessarily damaged her credibility. People often forgive but rarely forget these character lapses.

    “..damaged her credibility…”? I don’t think so – she’s been the focus of some rather unsavoury attention, and it may be that some people that involve themselves in this sort of stuff will hope for some residual mud-sticking.

    But credibility has a large subjective component – you seem comfortable in your odd incorporation of other people’s editorialising into your world view and your rather contrived misinterpretations (“it’s now all too easy for me to think that the original paper was indeed fundamentally flawed…” really!?). As far as credibility goes though, I would look at this more objectively – Dr Gergis has published 15 or 16 additional papers since submitting the original reconstruction, has worked constructively within several research teams who obviously consider her to be an excellent researcher and collaborator, has taken great effort in reassessing the original reconstruction to ensure it’s validity and has been courageous enough to put this through multiple additional rounds of editorial critique and to do all this quite openly.

    She seems to be a respected and gifted scientist – I expect she’s learned quite a bit from this episode which will stand her in good stead in the future and no doubt will comfortably survive the sneering from some quarters!

  91. Dikran Marsupial says:

    warofthewolds “DM: What do you call a man in control of his cognitive biases? A. deluded”

    no, that would be someone who thought they didn’t have them.

    “I mention the jury analogy because as we all know, expert witnesses regularly offer contradictory evidence”

    Goalpost shift, contradictory evidence is not the same thing as “behavioural indicators”.

    “Gergis has been somewhat economical with the truth in her article”

    Rashomon.

    “Although illogical to extrapolate that tendency to everything she said, she has unnecessarily damaged her credibility. People often forgive but rarely forget these character lapses.”

    In my experience (having published a paper with a mistake in it and publishing an corrigenda), people who were hostile to the finding in the first place were hostile to the corrigenda as well, and being open about admitting the mistake did not make a difference to the loss of credibility in those quarters. As Gergis rightly says, it is because blogs tend to be partisan and based on rhetoric rather than evidence, so you can talk about Gergis’ credibility all you like, but it makes no difference, what matters is whether the revised paper is correct or not, and all else is a distraction.

    “DM: “judged the case based on the evidence” well that would be a novel approach.”

    No, actually it wouldn’t. It is a reasonable expectation that cases should be judged in that way, and it is part of the responsibility of the judge to give the jury guidance on such matters and make sure they are not unduly biased by the process of the trial. So that is another soundbite.

    “Yes it is an uncharitable extrapolation (nice phrase) and was intended to be so for effect. ‘It’s all too easy’ is of course, the point.”

    In which case you ought to have pointed out that it is an irrational extrapolation from cognitive biases that a rational person should supress rather than promulgate. I did notice the “its all too easy”, it also “all to easy” for me to interpret that as being there for “plausible deniability”, it cuts both ways. Rashomon. ;o)

  92. War,

    I mention the jury analogy because as we all know, expert witnesses regularly offer contradictory evidence in court and for the judge/jury, credibility is the marginal difference.

    That’s why consensus studies are important. The reason we are regularly exposed to contradictory evidence is because the minority who are dismissive are over-represented. If experts were chosen randomly from all possible experts, then the evidence presented would probably appear far less contradictory than it currently does.

  93. Dikran Marsupial says:

    “I mention the jury analogy because as we all know, expert witnesses regularly offer contradictory evidence in court and for the judge/jury, credibility is the marginal difference. ”

    If there is only a marginal difference, of course, then the judge and jury shouldn’t draw any strong conclusions either way – duh!

  94. Dikran Marsupial says:

    ““I mention the jury analogy because as we all know, expert witnesses regularly offer contradictory evidence in court and for the judge/jury, credibility is the marginal difference. ”

    BTW I realise I misread war’s comment, so I apologise for that. However my point remains, credibility may be the marginal difference, but one would hope that credibility in expert witnesses is not based on “behavioural indicators”, but on objective assessments of their expertise, e.g. publication record. Otherwise you are just giving more or less free reign to your cognitive biases, rather than examining/suppressing them as a judge/jurist should.

  95. DM: “If there is only a marginal difference, of course, then the judge and jury shouldn’t draw any strong conclusions either way – duh!”
    If only. Typically, where two equally well qualified experts (Medical for example) disagree it is beyond the competency of the judge/jury member to distinguish the most reliable account. Reasonably strong evidence may support both positions. It then comes down to credibility. For the judge, who performed best under cross examination perhaps. For the jury, how somebody simply looks, speaks or dresses can make the difference. Sad but true.

    ATTP:That’s why consensus studies are important.
    Yes, I would agree with that in general with appropriate caveats. As a CAGW sceptic, I am not convinced that the current scientific consensus is based upon sufficient evidence to carry the weight it requires to persuade me. But I do recognise that a consensus exists and I take that into consideration.

    DM: “novel approach . . .So that is another soundbite.”
    Actually it was sarcasm aimed at the legal system. Presumably you didn’t get that but I thought it obvious. My bad.

    DM: “you can talk about Gergis’ credibility all you like, but it makes no difference, what matters is whether the revised paper is correct or not, and all else is a distraction.”
    Well, yes and no. I agree that the paper should stand or fall on its merits. On that, I note that Mosher’s prediction above is doing well so far but it’s early days yet. I disagree that ‘all else is distraction’ since the article piece is simply spin and it’s precisely that which most casual observers will ever stumble across and subsequently form their opinions on. Of course it’s important.

    Anyway, thanks for your thoughts DM (and the link to Roshomon) and for the blogpost ATTP. I’ll check back and read responses but probably won’t post again on this thread since it’s getting old. I’m sure your time is better spent discussing the science rather than arguing with an old woodcutter like me.

  96. “if only. Typically, where two equally well qualified experts (Medical for example) disagree it is beyond the competency of the judge/jury member to distinguish the most reliable account”

    Indeed, that is my point in which case they shouldn’t make the distinction, they should admit their lack of competence and not draw strong conclusions. As I indicated, this not exactly rocket science, it ought to be common sense. If they try to make distinctions based on “behavioural indicators” that is irrational and just encouraging their cognitive biases instead of dealing with them.

    “since the article piece is simply spin”

    Somewhat ironic that after all has been said you are still insisting on such an uncharitable interpretation. You have also ignored the substantive point, which is that if you do just admit your error (as I did) those who are “on the other side” don’t give that any credit either, so at best Gergis was on a hiding to nothing. Those looking to maximise the fault will continue to do so, no matter what is written. That is why this is a distraction. I know this from (not particularly pleasant) personal experience.

    Personally I think the “golden rule” is a better approach. As I have demonstrated, it is easy to interpret what you have written in an uncharitable manner, so if you want to treat what Gergis wrote as “simply spin”, then you need to realise that what you have written is not obviously different.

  97. Going back to what steven wrote

    “A) one side tends to get hyperbolic about the error.. all science is broken.. crisis in science
    B) one side tends to call all such things “typos”
    C) and then they refuse to give credit for finding what they argue is a trivial error.”

    Yep, Rashomon. “simply spin”etc. is (A), saying that it is just a typo is (B) (it is both imprecise and hyperbole). Both sides need to acknowledge there is a bit of “Rashomon” going on, and both should aim at being a bit more reasonable about things, and preferably concern themselves with the science.

    In my experience C makes no difference, I gave credit to those who pointed out the error, did it make any difference? No, at least not with those trying to avoid discussing the science.

  98. angech says:

    Gerghis, the paper, as opposed to Gerghis the person, is it any good?
    Some criticisms mentioned here suggested Dikran would not support it, the paper, in its procedures.
    Does it achieve the aim of giving a verifiable 1000 year history of Southern Hemisphere temperatures.
    Is everyone happy with the trended method now used?

  99. Mark - Helsinki says:

    As McIntyre rightly pointed out
    “Gergis et al 2016 stated that they screened proxies according to significance of the correlation to local gridcell temperature. Law Dome d18O not only had a significant correlation to local temperature, but had a higher t-statistic (and correlation) to local instrumental temperature than:

    24 of the 28 proxies retained by Gergis in her screened network;
    either of the other two long proxies (Mt Read, Oroko Swamp tree ring chronologies);

    Nonetheless, the Law Dome d18O series was excluded from the Gergis et al network. Gergis effected her exclusion of Law Dome not because of deficient temperature correlation, but through an additional arbitrary screening criterion, which excluded Law Dome d18O, but no other proxy in the screened network.”

    What do you think the valid reason was for screening out Law Dome d18O? Gergis doesn’t give one

  100. Mark - Helsinki says:

    Also, data mining techniques between 2012 and 2016 are different, they didn’t just fix a “typo” as they claimed.

    As such new data mining technique is what required the whole overhaul.

    These seem to be inconvenient truths

  101. Mark,
    As I think I have pointed out elsewhere, if these are such obvious inconvenient truths, then Steve should – IMO – try to engage somewhere slightly more formal than a blog. Blog discussions can be fascinating, but they don’t really get the scutiny of more formal scientific discourse.

  102. angech says:

    Not fair.
    There are 2 issues here.
    One of which is the way and intent of the Gergis paper being redone.
    The other the result.
    There are inconvenient truths re both.
    The end result is a paper which provides the only summary of some of the available data for a Southern Hemisphere millennial temp reconstruction.
    It has been 3 years of data gathering and sorting with 4 years of rewriting to get it to a publishable level.
    The problem, in my eyes at least is a lack of recognition of the need to properly interpret and present the data.
    Not Gergis fault.
    She had supervisors.
    This could have been so good if done properly at the start and incorporated those inconvenient truths, whatever the outcome.

  103. Not Gergis fault.
    She had supervisors.

    Patronising? Joelle Gergis got her PhD in 2006.

  104. Chris says:

    There are inconvenient truths re both.

    Perhaps the “inconvenient truth” is that the reanalysis produces a result rather similar to the original reconstruction..?

    I guess a more pertinent question (that could have been asked any number of times over the past 10-15 years) is: if the McIntyre’s et al think there’s something wrong with the analysis, why don’t they publish there own? If I remember correctly these guys did actually do this once (re Antarctic warming) and came up with pretty much the same conclusion as the paper they trashed so vigorously on their blogs…

    Worth noting that Dr Gergis has published another 15 or so papers since submitting the original reconstruction – she seems to be a gifted and respected scientist and research collaborator.

  105. This is the key question, IMO

    if the McIntyre’s et al think there’s something wrong with the analysis, why don’t they publish there own?

  106. That question has a simple answer: they would not like the results of their own analysis.

  107. MartinM says:

    Also, data mining techniques between 2012 and 2016 are different, they didn’t just fix a “typo” as they claimed.

    They didn’t claim any such thing. That’s literally the exact opposite of what they claimed.

  108. -1=e^iπ says:

    “if the McIntyre’s et al think there’s something wrong with the analysis, why don’t they publish there own?”

    I think that a lot of the data he would need in order to do his own analysis isn’t publically available. For example, look at the difficulty he went through to get law dome data: https://climateaudit.org/2016/08/03/gergis-and-law-dome/

  109. Magma says:

    That question has a simple answer: they would not like the results of their own analysis. — Victor Venema

    There is also the question of effort, competence and the hurdles of the peer review process. At an age where productive researchers have 100+ publications, McIntyre has three or four largely discredited publications with the same academic co-author. And I remain to be convinced that he was a standout in his “day job” either.

  110. -1,
    I don’t think that that is true. There is always going to be some data that is harder to get than others, but if you’re going to make strong claims about the results in a particular field then, ideally, you should be doing your own analyses, not simply highlighting possible problems without really illustrating the significance of these issues.

  111. -1=e^iπ says:

    I agree, it would be nice if McIntyre came out with his own analysis. However, pointing out flaws in such as with various work by the PAGES 2K group seems useful, especially as researchers have responded to some of his findings in the past. From what I can tell, the exclusion of Law Dome data does not seem well founded.

  112. Mark - Helsinki says:

    CA concludes “The Law Dome d18O series has stronger statistical relationship to gridcell temp than 24 of 28 “passing” proxies or either of the long tree ring series used as long proxies in Gergis et al 2016”

    ATTP? “If steve thinks” is not an argument, it is just arm waving. Lots of that in Gergis et al

    Any of you can confirm the above in this post. That’s how it works, go check if the screened out Law Dome does have a higher t stat than most of the proxies retained.

    Then you can make a counter argument, not talk about “what steve should do”. Such arguments are pointless

  113. Mark - Helsinki says:

    Given the site is called “climate audit”.. that’s a clue, they audit studies. Asking them to “do something else” in lieu of addressing their findings is imo a weak attempt to re frame the debate

  114. However, pointing out flaws in such as with various work by the PAGES 2K group seems useful

    I agree, but there is a difference between highlighting things that could be done differently, and actually illustrating that the overall result is flawed. The problem with what McIntyre does – IMO – is that he highlights some potentially genuine issues, but is not only over-confident in his own analysis, his commenters then cheerlead what he’s done and make all sorts of assertions about the overall result (and the field in general) that are not really warranted – IMO, at least.

  115. Mark - Helsinki says:

    MartinM says:
    August 6, 2016 at 1:43 am

    Also, data mining techniques between 2012 and 2016 are different, they didn’t just fix a “typo” as they claimed.

    They didn’t claim any such thing. That’s literally the exact opposite of what they claimed.
    __________________________________________________________________

    No they did change methods, and as such had to redo the whole paper.

    Clinging to “we spend 4 years correcting a type” idea doesn’t help.

  116. Mark,

    Given the site is called “climate audit”.. that’s a clue, they audit studies.

    Indeed, but then they need to recognise the significance of what is done there. Highlighting potential issues with some analysis does not immediately imply a major problem with the result. If you want to argue that the result is flawed, you need to either show that the analysis isn’t possible, or present an alternative analysis.

    Although I don’t think there is anything wrong with “auditing” scientific studies, I don’t think it is as useful, or worthwhile, as some seem to think. I think real progress comes from others trying to answer the same question using different methods, not people claiming to find problems with other people’s anaylses without ever actually showing the significance of these supposed errors.

  117. Any of you can confirm the above in this post. That’s how it works, go check if the screened out Law Dome does have a higher t stat than most of the proxies retained.

    Then you can make a counter argument, not talk about “what steve should do”. Such arguments are pointless

    So, Steve has shown that Law Dome has a higher t stat than the other proxies retained. The point of Gergis et al. was to produce a reconstruction of Australasia’s millenial temperature history, not what is the t stat of Law Dome.

  118. -1=e^iπ says:

    “I think real progress comes from others trying to answer the same question using different methods… The point of Gergis et al. was to produce a reconstruction of Australasia’s millenial temperature history, not what is the t stat of Law Dome.”

    I think McIntyre’s point is that an alternative analysis would be preferable. For example, the law dome data should have been included.

    If McIntyre had the data he needed then he could perform his own reconstruction.

  119. I think McIntyre’s point is that an alternative analysis would be preferable. For example, the law dome data should have been included.

    Yes, but the authors are entitled to disagree. That’s the beauty of doing scientific research; you don’t have to do what others say you should do, but they’re perfectly entitled to try and show that you’re wrong, or that they have a better method.

  120. -1=e^iπ says:

    @ATTP – Do you think there is any scientific justification for excluding the Law Dome data in the reconstruction?

  121. -1,
    Gergis et al. clearly argued that there was, but I certainly don’t know for sure. I particularly dislike the kind of question you’ve asked. I think Gergis et al. are entitled to argue that it should be excluded and McIntyre is entitled to argue that it shouldn’t. What do you achieve by asking me what I think, other than trying to put me into some kind of camp?

  122. -1=e^iπ says:

    They didn’t really argue that there was. All they did was apply a completely arbitrary and unjustified condition to exclude it ‘This comparison [of detrended proxy to detrended instrumental data] was only performed for cells containing at least 50 years of data between 1921 and 1990.’.

    The law dome data had a better t-stat despite a lower number of observations over that time period and if you used the full time period then there are at least 50 years of data to compare law dome data with instrumental data.

    They excluded one of their best data sets and 1 of 3 that had data prior to 1100. I don’t see any scientific argument that could justify this. Imo, either they made an honest mistake or it was intentional.

    “What do you achieve by asking my what I think other than trying to put me into some kind of camp?”

    You are the one claiming that their auditing isn’t useful. Seems pretty useful to me; inclusion of Law Dome data would give better results, especially prior to 1100.

  123. -1,
    Sorry, I’m not all that interested in moving the Law Dome discussion here, especially given that you’re now misrepresenting me. I didn’t say

    You are the one claiming that their auditing isn’t useful.

    I said “I don’t think it is as useful, or worthwhile, as some seem to think“. Of course highlighting alternatives, or highlighting problems with a method, can be sueful, but “auditing” alone doesn’t always indicate the significance of what has been highlighted; you do need to go and redo the actual analysis. What we’re interested in is the result being presented, not whether or not two people disagree about some detail of the analysis.

    This has not been illustrated, which is my point

    inclusion of Law Dome data would give better results, especially prior to 1100.

    If someone could redo the analysis with Law Dome included, that would be very interesting. That hasn’t – as far as I’m aware – been done.

  124. chris says:

    You are the one claiming that their auditing isn’t useful. Seems pretty useful to me; inclusion of Law Dome data would give better results, especially prior to 1100.

    Useful for what? Would inclusion of Law Dome data “give better results”? I expect it doesn’t matter, much in the same way as the ludicrous fuss about including (or not) a so-called “upside down” proxy data set didn’t matter in Mann et al’s PNAS temp reconstruction paper despite the rather unpleasant attempts at “auditing” that paper!

    There is hardly a study published that couldn’t be marginally improved in hindsight – the only reason for addressing what might be an improved methodology/analysis is if there’s an expectation of a materially different conclusion. Science moves on – Joelle Gergis has published around 15 papers since first submitting the paper we’re all discussing. No doubt one could “audit” each if these papers and find something to make a fuss about on a blog! (With enough “auditors” it might be possible to bring the whole scientific enterprise to a grinding halt 🙂 )

    I would say that’s the status of this work now – it’s published and like all published work will get the attention, and make the contribution it deserves according to its usefulness. If someone wants to redo the analysis using the Law Dome data then great – go for it. If “the auditors” were to do this and publish a paper then very likely we’d have two rather similar analysis that would give greater confidence to the use of these particular proxies and their interpretations. After all, that’s what The “auditors” did with the Antarctic proxy series following the Steig et al analysis and so we now have two studies that confirm the interpretation of significant warming in West Antarctica… (and several more since, but that’s by the by).

    Much, much better in science to do stuff rather than sit on the side-lines bitching about it! 🙂

  125. -1=e^iπ says:

    “Useful for what? Would inclusion of Law Dome data “give better results”? I expect it doesn’t matter”

    Given that it’s one of the best proxies, one of the only 3 prior to 1100, and shows very different behaviour prior to 1100, I suspect it would matter a fair amount.


  126. snarkrates says:

    Mark: “Given the site is called “climate audit”.. that’s a clue, they audit studies.”

    No. What they do is carry out a half-assed takedown of studies whose results (or authors) they don’t like. The goal is not to assess the quality of a study, but rather to provide a rationale for continued denial to other right-wing nutjobs. The scientific method already provides a method of assessing the quality of a study–building on that study by publishing new and independent studies. One suspects that McI has the statistical chops to actually do this. He simply refuses to submit his analyses to the rigor required to publish useful science.

  127. > I suspect it would matter a fair amount.

    We cannot build our dreams on suspicious minds.

    Work on it and report, -1.

    When will the Auditor audit satellite data or Tony’s stuff, btw?

  128. MartinM says:

    Clinging to “we spend 4 years correcting a type” idea doesn’t help.

    Then stop doing it. The only person who has suggested this idea is you. Gergis certainly didn’t; she said the exact opposite, as I already pointed out. She said that they could have taken the easy option and fixed the text of the paper, but instead opted to do a bunch of additional analysis.

  129. MartinM says:

    I think that a lot of the data he would need in order to do his own analysis isn’t publically available.

    Nope. All of the proxy data used in Gergis et al. is publically available through the NCDC, including the Law Dome d18O series..

  130. -1=e^iπ says:

    @ Martin, oh sorry my mistake.

  131. Dikran Marsupial says:

    “Given the site is called “climate audit”.. that’s a clue, they audit studies.”

    I would venture an audit should be unbiased and objective. If the auditor goes around insulting opponents and dismissing arguments as “arid pontificating”, then the audit isn’t an audit, it is just a review or critique. If you don’t properly consider the opposing arguments, it is not a systematic evaluation of the issue. Calling it an audit doesn’t make it an audit.

    That is not to say that CA doesn’t do something genuinely useful sometimes, just that they could do something more useful more often.

  132. Dikran Marsupial says:

    -1 “I think McIntyre’s point is that an alternative analysis would be preferable. ”

    Well perhaps a more productive approach would be to perform the analysis and publish a journal paper? That is what people normally do when they think an alternative analysis would be useful.

  133. Steven Mosher says:

    “When will the Auditor audit satellite data or Tony’s stuff, btw?”

    too funny
    a sly form of
    Tu quoque

    its even more silly since the key to good auditing is being a SME

    Mc is not a SME in temperature data. That’s why his late time out working with Tony he goofed.
    since then.. he sticks to what he knows. Proxies.

    Yes Virginia, people get to focus. They get to specialize.

    I will ask my Gastro Surgeon when is is going to start working on Ortho..

  134. Joshua says:

    When will the auditor audit proxy analyses published by “skeptics?”

    Oh wait, there’s a problem there, isnt there?

  135. Dikran Marsupial says:

    ISTR CA was unable to see what was wrong with the Douglass et al paper (despite it being obviously wrong, given that the test would be almost certain to reject a perfect model ensemble). That could be indicative of bias, but it could also be evidence of being only human and getting things wrong every now and again.

  136. Yes Virginia, people get to focus. They get to specialize.

    Yes, but whether or not what they do is taken seriously is not up to them.

    I will ask my Gastro Surgeon when is is going to start working on Ortho..

    Alternatively, you might ensure that you choose a Gastro Surgeon who actually does Gastro surgery, rather than one who spends all their time criticising other Gastro Surgeons.

  137. angech says:

    ATTP
    “Patronising, Joelle Gergis got her PhD in 2006.”
    She was the lead researcher on the paper.
    David Karoly was the lead author hence one of her supervisors on the paper.
    Karoly made 2 elementary errors.
    The first was in setting up the paper with the past data compared to a long term trend from 1920 to 1990.
    The paper was then presented as using the detrended data but actually used the trended data and as the lead author along with others, he missed that.

    The second error by David was still in the new paper which took four years to correct. There was a lot of reworking done to get the data with long term trending used to match the original paper.
    This reworking by necessity included Karoly’s error and was referred to by Mosher in a cryptic appeal to Dikran earlier.
    As such it might invalidate the new paper.

    That is why I said it is a shame the parameters for the study were not set out and followed in a routine way which would have produced a routine scientific scientific result which no skeptic would have been able to have been upset with.

  138. angech,

    She was the lead researcher on the paper.
    David Karoly was the lead author hence one of her supervisors on the paper.

    What are you talking about? If paleo is anything like my field, then the first-author is the lead researcher and lead author. The paper is Gergis et al. Not Karoly et al. Also, once someone has completed their PhD they’re typically no longer in a supervisor/supervisee relationship. The lead author on a study is never – IME – regarded as the supervisor of all the others on the paper.

    That is why I said it is a shame the parameters for the study were not set out and followed in a routine way which would have produced a routine scientific scientific result which no skeptic would have been able to have been upset with.

    Firstly, how do you really know that they weren’t? I realise that you think they weren’t, but that doesn’t mean that they weren’t. Assertions are not that convincing. Secondly, as far as I can tell there probably isn’t any way to do a multi-proxy study (millenial temperature reconstruction) that won’t upset “skeptics”. I think you’re asking for the impossible.

  139. -1=e^iπ says:

    “as far as I can tell there probably isn’t any way to do a multi-proxy study (millenial temperature reconstruction) that won’t upset “skeptics”.”

    That’s true. Although I think people McIntyre could be satisfied.

    Anyway, the 2nd millennium reconstruction is probably fine.

    It’s the first millennium where the results are more questionable.

  140. That’s true. Although I think people McIntyre could be satisfied.

    I’ll believe it when I see it.

    Anyway, the 2nd millennium reconstruction is probably fine.

    It’s the first millennium where the results are more questionable.

    Which millenium is which?

  141. angech says:

    ATTP the article linked by you to Gergis above (second link) stated Prof Karoly as the lead author and further in the article Joelle Gergis as the lead researcher.

    “Update October 2012: The manuscript “Evidence of unusual late 20th century warming from an Australasian temperature reconstruction spanning the last millennium” by Joelle Gergis, Raphael Neukom, Ailie Gallant, Steven Phipps and David Karoly has been re-submitted to the Journal of Climate and is being reviewed again.

    It was originally put on hold after the authors identified an inconsistency between the methodology described and the methodology that was ultimately used to produce the findings.

    The lead author, Professor David Karoly, told The Conversation: “The actual method … included the long-term trend in the temperatures over the period from 1920 to 1990. The manuscript describes the method as having removed the long-term trend. So there’s an inconsistency between what is written in the manuscript and the method that was actually used.”

    The scientific method was not done properly, Mosher explained why, resulting in an unfortunate inconsistency in the new paper.

  142. angech,
    I’m not quite sure where I linked to that article, but if it does indeed refer to Karoly as the lead author, then I stand corrected. That still doesn’t really make him the supervisor (in the sense that it is typically used in academia) of the other authors.

    The scientific method was not done properly, Mosher explained why, resulting in an unfortunate inconsistency in the new paper.

    As much as I respect Mosher’s genuine scepticism, this doesn’t mean that the scientific method wasn’t done properly; people asserting things doesn’t make them true (why isn’t this obvious?). Complaining about details on blogs isn’t really part of the scientific method either. Nothing intrinsiclly wrong with it, but it probably won’t have much impact.

  143. -1=e^iπ says:

    “Which millennium is which?”

    first millennium – 0-999 AD
    second millennium – 1000-1999 AD

    Again, only 2 data sets cover the first millennium, unless they were to include law dome.
    Of course Gergis’ primary conclusions cover the second millennium.

  144. Steven Mosher says:

    “Alternatively, you might ensure that you choose a Gastro Surgeon who actually does Gastro surgery, rather than one who spends all their time criticising other Gastro Surgeons.”

    I’ve had long discussions about this with Steve.

    Speaking metaphorically he opened the patient.. saw a deadly cancer that was inoperable
    and quietly sewed the patient back up.

    Its akin to the people ( purists) who look at temperature data and want to start over with a clean slate because they argue that its beyond fixing

    I dont think these two groups of people (Purists vs. do the best you can) can talk to each other.

  145. Steven Mosher says:

    I will say this on the Audit metaphor.

    CA plays the ball and the man.

    then when people defend the man, people bitch that they are not playing the ball.

    Angech, it gets directly to the question Dr. Rice was asking

    What is the purpose?

    Suppose that your Math is wrong and you are ugly

    I have a choice about what to say

    A) angech’s math is wrong, here is a better way let me show you
    B) angech’s math is wrong. and nobody in his field will challenge him, so I am forced to
    C) angech’s math is wrong and he is ugly …. the crowd roars they are all ugly!!!

    well if your purpose is improving things….

    My experience– I’ve played the ball and the man a LOT.

    it doesnt help. feels good. but its not exactly the scientific method.. and I use that term broadly.

  146. Steven Mosher says:

    “When will the auditor audit proxy analyses published by “skeptics?”

    Oh wait, there’s a problem there, isnt there?”
    ##########################
    You need to read more history. who came up with the idea of having CA critque a skeptics reconstruction?

    9 years of history is not that much. I will tell you what bender told me. Shut up and read everything before commenting.

    https://climateaudit.org/2007/11/15/craig-loehle-reconstruction/

    Posted Nov 15, 2007 at 1:51 PM | Permalink
    RE 565. Congratulations craig.

    I say you need to run the GAUNTLET here at CA and have people throw tomatoes at your paper.
    I cannot imagine anything more exhilerating than having the CA crowd throw tomatoes at my work.

    I’ve downloaded yours. Sad to say my tomatoes are small but I’ll throw what I have when I get a chance.

    ############

    Posted Nov 15, 2007 at 2:58 PM | Permalink
    Re: Steven Mosher throwing tomatoes at my reconstruction (see toward end of Unthreaded #24): Thanks to CA I was warned about certain data sets, and warned also about the variety of tomatoes that can be thrown, so I’m not too worried about that. I can imagine that RC will turn up their noses at the proxies I used, since they are addicted to bristlecones and I didn’t use any fancy stats. By the way, GRL sent it back unreviewed because they were tired of seeing reconstructions.

    bender

    https://climateaudit.org/2007/11/15/craig-loehle-reconstruction/#comment-117918

    Rob Wilson

    https://climateaudit.org/2007/11/15/craig-loehle-reconstruction/#comment-117961

  147. Willard says:

    > a sly form of / Tu quoque

    You call that “sly”? Here’s sly::

    To my recollection, I haven’t seen any criticism or condemnation of these lawsuits.

    https://climateaudit.org/2016/07/21/joelle-gergis-data-torturer/#comment-770016

    We never saw any criticism or condemnation of the satellite data and code.

    We never saw criticism or condemnation of Tony’s crap.

    We barely saw criticism or condemnation of Wegman’s report.

    The list goes on and on, since the Auditor never really paid any due diligence to contrarian crap.

    If auditors allow themselves to be free to audit whatever they fancy, they should let otters do the same.

  148. Steven,

    Speaking metaphorically he opened the patient.. saw a deadly cancer that was inoperable
    and quietly sewed the patient back up.

    Well, yes, this does seem to be what is implied. It’s certainly possible that the issues are so great that we can’t actually even do the analysis, but this needs to be demonstrated and would suggest that not only do we know nothing about millenial temperatures, we can know nothing. I think most people regard attempts to understand something as preferable to claims that it isn’t possible.

  149. If auditors allow themselves to be free to audit whatever they fancy, they should let otters do the same.

    Indeed, which is one reason I think he should submit his current criticism to something more formal than his own blog. Thay way not only does it stand a great chance of having actual impact, but it also becomes more likely that it will be properly scrutinised by others.

  150. -1=e^iπ says:

    Would be nice if after PAGES 2K update comes out and people have a chance to look through it, if a group could do a similar analysis but with a more statistically rigorous methodology. Sort of like what BEST did with temperature data.

  151. angech says:

    Thanks ATTP,
    The quoted relationship was in the Conversation article re David Karoly.

    “Steven Mosher. July 23, 2016 at 4:55 pm
    The problem is it authorizes the use of several suspect practices.
    A) comparing a proxy to multiple temperature fields to find a correlation
    B the proxy predicts temperature. Unphysical is bad.
    c) screwing up your calibration and verification times.
    Tho it don’t change the physics.”

    C is the money issue.
    There is limited blog auditor style evidence that the second study has a flaw which may need it redacted again.
    It would be a shame to see this happen for a similar reason to the first paper.
    Mislabelling time periods possibly used in this case. 1920 -1990.
    Not my maths.

  152. Chris says:

    There is limited blog auditor style evidence that the second study has a flaw which may need it redacted again. It would be a shame…

    I doubt it, notwithstanding your crocodile tears…

    Did all that bluster about retractions on “auditing” blogs about “upside-down” proxies or Antarctic temperature analyses etc. etc. ad nauseum have any effect other than bolstering the courage of the febrile followers of that unpleasant stuff?

    There is a very basic issue of science here. Gergis et al has been published following a rather serious reassessment and editorial review process. It’s one of a very large number of proxy studies (there are other S hemisphere proxy analysis too). I’m comfortable with the likelihood that it’s a valid analysis of proxy series for her study area. However if anyone is concerned that the results may be meaningfully different using a different type of analysis then they should do this alternative analysis and publish it.

    That’s the bottom line. Either you’re interested in the science and so you should do the appropriate analysis to address your concerns – or you’re interested in hounding whoever happens to be the current target for mud-slinging… happily for Gergis, the spotlight will no doubt be shifted to the next unfortunate scientist in due course 🙂

    Interestingly one could come up with a group of papers that by objective measures really should have been retracted – since you’re obviously interested in the possibility of retracting scientific papers we could discuss some of these..

  153. -1,

    Would be nice if after PAGES 2K update comes out and people have a chance to look through it, if a group could do a similar analysis but with a more statistically rigorous methodology. Sort of like what BEST did with temperature data.

    I don’t know if this is what you meant by this, but there’s a standard blogosphere narrative that what climate science needs is the injection of a bunch of independent statisticians who will suddenly do things properly and correct all the statistical errors being made by current climate scientists. Here’s my view on this

    1. There’s nothing wrong with including people from other disciplines who have relevant expertise, but it’s not free; someone has to fund it.

    2. If a bunch of statistician are interested in climate science, there’s nothing stopping them from getting involved.

    3. How long before n “independent” statisticians are no longer independent? Do we have to regularly inject new “independent” statisticians?

    4. Even though climate scientists do sometimes blunder when carrying out statistical analyses, there are many who are very skilled statisticians and a number who have formal training in statistics.

    5. There are plenty of examples of “independent” statisticians thinking they can jump into another field and do things better than those already in the field, making all sorts of silly mistakes because they didn’t bother to familiarise themselves with the field in question.

    So, there’s nothing wrong with more people getting involved, but it’s not some kind of panacea and there’s no real guarantee that some group of statisticians won’t make silly blunders of their own.

  154. angech says:

    Chris
    “There is a very basic issue of science here. Gergis et al has been published following a rather serious reassessment and editorial review process.”
    She could have “taken the easy way out and just correcting the single word in the page proof, ”
    Her work was so good originally that just a single word change could fix it,
    Yet it took 4 years of work to correct it,
    are concepts that do not coexist together.

    The problem though is that after 4 years of work, which involved a lot more than changing one word, they changed things they should not have changed.

    You said yourself
    “. Either you’re interested in the science and so you should do the appropriate analysis to address your concerns” and,
    “I’m comfortable with the likelihood that it’s a valid analysis of proxy series for her study area.
    That’s the bottom line, comfort and likelihood are not substitutes for interest or analysis.

    1920 -1990 should be equivalent in both studies, original and redo, but there is a concern that the analysis shows they are apples and oranges and quite uncomfortable.

  155. angech,
    I’m not even sure what this supposed problem even is.

  156. BBD says:

    I’m not even sure what this supposed problem even is.

    Politics.

  157. Chris says:

    …but there is a concern that the analysis shows they…

    well you’re certainly showing a lot of concern on this thread…

    if you’re so concerned you should redo the analysis yourself according to how you believe it should/might have been done. Until you or someone does that I’m pretty comfortable with Gergis’s updated analysis. After all we’ve been here many times before (so much “concern” so little substance)…

  158. “2. If a bunch of statistician are interested in climate science, there’s nothing stopping them from getting involved.”

    which is what I did.

    It is somewhat ironic that -1 is calling for better statistical methodology after repeatedly making elementary statistical errors on another thread (for instance the classic p-value fallacy). As it happens, I don’t think the standard of statistics is any lower in climatology than in other branches of science. I suspect if people thought that astronomy (for example) was likely to have a substantial influence on peoples tax burden, the spotlight would be there instead. The misinterpretation of Prof. Jones’ answer to the famous loaded question about statistically significant warming suggests that he understands statistical hypothesis tests rather better than his critics.

    “Sort of like what BEST did with temperature data.”

    Has BEST satisfied climate skeptic objections about surface temperatures or homogenisation? Why expect it to be any different in this case?

  159. angech wrote “Her work was so good originally that just a single word change could fix it,
    Yet it took 4 years of work to correct it,”

    Misrepresentation. A little more than a single word change (yes, so there was a bit of hyperbole) could have fixed the paper but not the work, at least not to Gergis’ satisfaction, which is why they took the long road, rather than the simple fix.

  160. 3. How long before n “independent” statisticians are no longer independent? Do we have to regularly inject new “independent” statisticians?

    They are independent until they get the politically inconvenient answer. Like Anthony Watts was going to support BEST no matter the outcome and immediate condemned them when they got the politically inconvenient answer.

    In the mean while, while the “independent” statisticians are working on it, you have an excuse to delay solving the problem and therewith making the problem bigger and harder to solve.

  161. VV indeed in the acronym FLICC used to characterise rhetorical strategies used in discussions of climate, the “I” stands for “impossible standards”. The “independent statisticians” is part of this, and as the BEST example shows it may just be a stalling technique when the standard is reached (I’d say it ought to be “unreasonable/infeasible/inappropriate standards”, they don’t have to be genuinely impossible for it to be a useful strategy)

  162. Victor,
    Exactly. When the results turn out to be essentially the same as before, there’ll be complaints that they’ve been corrupted by the system.

  163. Willard says:

    > [The Auditor] opened the patient.. he saw a deadly cancer that was inoperable and quietly sewed the patient back up.

    One tiny problem with that reading is that nothing in the Auditor’s hostile takeover prevents the Auditor to present provisional and exploratory reconstructions:

    [Paul Dennis, distancing himself from his ClimateBall career] New samples can then be collected from a range of proxies from suitable sites and a climatic record determined that involves no post hoc data screening or selection. This would either confirm or refute the exploratory studies. Until such research is done all present reconstructions should be treated as provisional and merely exploratory in nature.

    [The Auditor, jumping onto this econometric hostile takeover] I agree 1000% with your statement.

    The Auditor’s business model is more akin to a hostile takeover than an audit.

  164. “and quietly sewed the patient back up.”

    quietly?

  165. Joshua says:

    This is beautiful, from Steve McIntyre:

    From my perspective, I referred to her false narrative mainly in passing and did not attempt a detailed exegesis.

    It’s always interesting to note the use of qualifiers such as “from my perspective,” and “mainly in passing.”

    Similar qualifiers as rhetorical devices (favorites of those who like to employ the Chewbacca defense and folks who hang at Lucia’s, btw) are “you are essentially saying” and “basically, what you said is….”

    So let’s look at the outcome of Steve’s efforts against his “perspective” of whether or not “mainly” speaking to “her false narrative” is peripheral to his main focus..

    What % of the focus in the discussion at Climate Audit, or here for that matter, is on the Gergis et al., science as compared to the falseness or veracity of Gergis et al.’s “narrative?” To what extent does the “mainly in passing” aspect of Steve’s focus contribute to the atmosphere of personal animus?

    I see this as one of Steve’s characteristic MO’s. Put something out there where it’s incredibly easy to predict that it will stimulate and channel personal animus, and then shirk accountability for having done so under a cloak of plausible deniability.

    And then, of course, the self-victimization comes into play.

    1) Poison the well.
    2) Complain that the water tastes bad.

    Sameosameo.

  166. Marco says:

    “There are plenty of examples of “independent” statisticians thinking they can jump into another field and do things better than those already in the field, making all sorts of silly mistakes because they didn’t bother to familiarise themselves with the field in question.”

    McShane & Wyner comes to mind.

    The auditing process at CA of that paper was certainly not as critical as it should have been (e.g., McIntyre should have noted the frequent misrepresentations of what was done in MBH98/99). Fortunately, we still have Eduardo Zorita:
    http://klimazwiebel.blogspot.dk/2010/08/mcshane-and-wyner-on-climate.html

  167. I was at a scientific conference with a lot of independent statisticians in June, the International Meeting on Statistical Climatology. It is quite well 50% statisticians and 50% climatologists.

    http://13imsc.pacificclimate.org

    As long as they are not harassing climatologists or showing that the climate is not changing (much), they will not be considered true independent statisticians by WUWT & Co. That is just a fact of live.

  168. Victor,
    I’m guessing that they probably don’t even know that such meetings already take place (how can they, the data still shows warming) and – if they do – would argue that 50% statisticians isn’t enough.

  169. JCH says:

    Is one statistician enough to prove all the other statisticians are wrong, or are 100% of the insufferable sobs always right?

  170. The “no statistically significant warming so there must be a pause” is enough to show where the rigorous statistics doesn’t reside.

  171. Willard says:

    > Is one statistician enough to prove all the other statisticians are wrong, or are 100% of the insufferable sobs always right?

    An alternative hypothesis is that it’s a cultural thing, e.g.:

    As a field based scientist (geologist) [my emphasis] with a strong experimental background (experimental mineralogy and isotope geochemistry) such an approach is anathema to me. I think it is the same for most others with backgrounds in the physical sciences and engineering. […] The fact remains that however you dress it up data mining it is still data mining and exploratory in nature. It tells us little, or nothing about the climate of the past until confirmed by robust scientific studies.

    Considering that it’s been a while since data mining has been reconsidered and that it is even a discipline in its own right nowadays, chances are that statistical purism will subside a bit with the next generation of auditors.

    For more about the geological perspective, Bender’s advice to “read the blog” still has currency.

    Physicists don’t own the monopoly on obnoxiousness.

  172. Willard says:

    Oh, and in passing, the Auditor’s tackle:

    ATTP celebrates the obstruction as Climateball.

    is countered by the Auditor’s own spit ball:

    You also have a bad habit of paraphrases that distort the original point.

  173. Magma says:

    @ Willard: I’m not sure Dr. Dennis constitutes an ideally objective critic of the many climate scientists falling short of his own personal standards. Though he does exude concern by the bucketful, doesn’t he?
    https://www.theguardian.com/environment/2010/feb/04/climate-change-email-hacking-leaks

  174. Magma says:

    Quite a gem in McIntyre’s 2005 geological perspective post. He must have thought so as well, since he emphasized it in bold in the original: So one would certainly not be able to say that 1998 was the warmest year in the past 50 million years. – not even close.

    Thereby refuting the warmist crowd of — hmm — none, who made that claim.

  175. Willard says:

    > 50 million years

    I think it’s more milll-yun years, Magma. But interesting nonetheless, as may have exclaimed PaulD!

  176. Dikran Marsupial says:

    “The fact remains that however you dress it up data mining it is still data mining and exploratory in nature.”

    Well data mining is more or less by definition exploratory. However not all statistical analysis is data mining, and before data mining there was exploratory data analysis (EDA) championed by some of the best statisticians around at the time, such as John Tukey. There is nothing wrong with EDA or data mining, provided that you know that is what you are doing and the issues involved.

    Another example of saying something that is technically correct in order to impley something that is complete baloney.

  177. Dikran Marsupial says:

    JCH now now, that’s enough of that, if you don’t mind! ;o)

  178. angech says:

    “Misrepresentation. A little more than a single word change (yes, so there was a bit of hyperbole) could have fixed the paper but not the work, at least not to Gergis’ satisfaction, which is why they took the long road, rather than the simple fix.”

    The satisfaction needed was that of the editor.

    You cannot do the original paper, and submit it without being satisfied that it was up to the standard required of the journal.
    The problem therefore is partly on her reviewers, who all missed the simple single word change and passed it as fit for publishing.
    Surely an issue in this age of peer review.
    Gergis having said that she had detected this simple mistake should simply have corrected it.

  179. angech,
    You still haven’t explained (or I can’t find where) what this “single word change” is.

  180. Dikran Marsupial says:

    angech wrote “The satisfaction needed was that of the editor.”

    No, actually it is quite important for the authors of a paper to be satisfied with it, as their academic reputation depends on it. That is why Gergis took the time to go back and perform the study the way they thought it ought to be done. Only bad academics think that volume of publications is superior to quality of publications.

    “You cannot do the original paper, and submit it without being satisfied that it was up to the standard required of the journal.”

    Rubbish, of course you can, it is called “making a mistake”, such as accidentally having a flag set to the wrong value or multiplying by a constant when you should have divided.

    “The problem therefore is partly on her reviewers, who all missed the simple single word change and passed it as fit for publishing.”

    This is idiotic, peer review is only a basic sanity check, reviewers don’t have the time to find every error in every piece of work they review.

    “Gergis having said that she had detected this simple mistake should simply have corrected it.”

    O.K. so you still don’t understand the issue, try reading the comments above in more detail, this has already been covered more than once.

  181. angech says:

    Dikran
    New paper
    “For predictor selection, both proxy climate and instrumental data were linearly detrended over 1931–90. As detailed in appendix A, only records that were significantly (p < 0.05) correlated with temperature variations in at least one grid cell within 500 km of the proxy’s location over the 1931–90 period were selected for further analysis-"

    Old paper
    "The lead author, Professor David Karoly, told The Conversation: “The actual method … included the long-term trend in the temperatures over the period from 1920 to 1990."

    Problem is obvious.
    I am sorry there is a problem.

  182. Why is it obvious? David Karoly is talking about the old paper, and you’ve quoted the new paper. The new paper doesn’t need to be consistent with what someone said about a version that was withdrawn a few years before the final version appeared.

  183. Dikran Marsupial says:

    angech it is obvious once the new paper has been published (and the nature of the error explained by the authors). Reviewers are not generally issued with a time machine to help them with their reviewing.

  184. Dikran Marsupial says:

    angech has also dodged the issue of why they authors chose not to go with the quick fix but instead took the trouble to correct the analysis.

  185. angech says:

    “How a single word sparked a four-year saga of climate fact-checking and blog backlash
    July 11, 2016 6.38am AEST”
    was the headline description in the Conservation of Dr Gregis’s article
    The single word referred to was “detrended”
    Gergis said
    “While the paper said the study had used “detrended” data – temperature data from which the longer-term trends had been removed – the study had in fact used raw data. When we checked the computer code, the DETREND command said “FALSE”

    The study may also have explained why they chose to use detrended data instead of raw data in some detail in the first place, which made it hard to then use the study of raw data, which it was, and could have been done by removing the word detrended.
    This would have made the study correct, no need for change to detrended data, except for the fact that they had spent a lot of time saying detrended data was better, which it obviously was not.

    The new study uses the raw data but changed a lot of the methodology criteria and some data samples.

  186. The new study uses the raw data but changed a lot of the methodology criteria and some data samples.

    You seem to be suggesting something nefarious. The point is simple. If you find a mistake before a paper has even appeared, you would normally tell the editor of the journal. You might think it is a minor issue; they might disagree. You may be encouraged (forced even) by the editor to withdraw your paper, check it thoroughly, and re-submit. You may discover – upon doing so – that the paper would be better with a different/updated methodology. You redo the analysis and resubmit the paper which is now quite different to the original version. None of this means that it wasn’t something that appeared trivial that lead to all these changes.

  187. Dikran Marsupial says:

    “The new study uses the raw data but changed a lot of the methodology criteria and some data samples.”

    was the methodology in the new paper better or worse than in the old paper?

  188. Willard says:

    > [D]ata mining is more or less by definition exploratory.

    This indicates that PaulD may use the term in a derogatory manner. The terminology shifted at the Auditor’s over the years, but what has been dogwhistled more or less remained the same. Witness this D’Arrigo ClimateBall ™ episode:

    I don’t have the exact words here. (I’ll edit it if I get better notes.) But, for certain, D’Arrigo put up a slide about “cherry picking” and then she explained to the panel that that’s what you have to do if you want to make cherry pie. The panel may have been already reeling from the back-pedalling by Alley and Schrag, but I suspect that their jaws had to be re-lifted after this. Hey, it’s old news at climateaudit, but the panel is not so wise in the ways of the Hockey Team. D’Arrigo did not mention to the panel that she, like Mann, was not a statistician, but I think that they already guessed.

    https://climateaudit.org/2006/03/07/darrigo-making-cherry-pie/

    Note the “Hockey Team” jab, only “in passing.”

    Those who study data mining will appreciate not being worthy of the “statistician” epithet by the Auditor. Not that it matters – after all, they’re just a bunch of cherrypickers.

    ***

    Speaking of statisticians reminds me of Auditor’s programme of rewriting paleoclimatology using the Browne and Sundberg:

    [T]his is the sort of thing that I’m inclined to work up for publication in a technical publication. It’s something that I’ve been working at on and off for a long time and only recently got a secure foothold on Brown and Sundberg. The approach that this leads to is clearly fresh and distinctive, and will lead to new perspectives without being argumentative.

    https://climateaudit.org/2008/07/30/brown-and-sundberg-confidence-and-conflict-in-multivariate-calibration-1/

    Notice the date. Notice the number of comments. Compare with the number of comments on the Gergis ClimateBall episode at the Auditor’s.

    Too bad the Auditor did not follow suit.

    ***

    > angech has also dodged the issue […]

    Angech peddles in the same concerns 1U did before him, but with a Moshpitian syntax.

  189. Magma says:

    Too bad the Auditor did not follow suit. — Willard

    He’s an auditor, not a sprinter. Jeez. Have some patience.

  190. Willard says:

    Perhaps I’d need to take a Ritalin even if I’m not a climate scientist.

    That said “in passing,” of course.

  191. Steven Mosher says:

    Angech.

    I hope you remember everything you write when and IF Watts 2012
    Finally makes it to print after more than 4 years.

    And recall, the only mistake was not using TOBS data.. a simple one line code fix.

    looks like they re ran the code, and the answer did not hold up.. so 4 years of data torture
    and I am still waiting for them to post their data from their 2012 web release.

  192. Steven Mosher says:

    “angech it is obvious once the new paper has been published (and the nature of the error explained by the authors). Reviewers are not generally issued with a time machine to help them with their reviewing.”

    For some journals they do ask you for all the old reviewer comments and your replies

  193. Steven Mosher says:

    angech

    “The problem is it authorizes the use of several suspect practices.
    A) comparing a proxy to multiple temperature fields to find a correlation

    I’m having second and third thoughts about bonferroni corrections

    First understand what one of the potential problems is.

    Suppose you have a gridded field suppose a grid 165,170, 15, 20
    (lon,lon, lat lat)

    Suppose your proxy is at 165,2, 15.1.. Lower left corner

    Do you look for a correlation in the grid it happens to fall into?

    But gridding ( ALA CRU ) is arbitrary. change your grid size, or origin and you have a proxy
    in a different grid cell. The problem is real. the solution is hard.

    and remember by applying the bonferroni correction you may increase your type II errors..

    and since all the grids are highly correlated does a simple bonferroni correction make sense?
    or is it too conservative?

    I have an idea for a new product. hmm. need to heal.

  194. Steven Mosher says:

    “One tiny problem with that reading is that nothing in the Auditor’s hostile takeover prevents the Auditor to present provisional and exploratory reconstructions:”

    Its not a reading. Its his position ( as expressed to me ) that much of the data is unuseable
    so you are basically LIMITED to doing EDA. The other problem is after having worked with the
    same limited number of proxies.. after a while.. you come to know the magic ones. that colors
    your screening approaches.

    hence his continued insistence on bringing proxies up to date.

    Basically they seem to be arguing that you could

    1. Screen proxies for correlation to temperature.
    2. Then go back and resample those proxies and bring them up to date.

    Note however that if someone only published EDA, they would complain about that as only
    being EDA.

    Schematically this is pretty simple.

    A) there are a limited number of proxies.. Spend a few years and you know them all. on sight.
    B) Your first analyst choice is the screening choice. which to use
    c) Your second choice is which meat grinder.
    d) your output is sausage.

    Its his opinion that ALL of the grinders amount to the same thing. some sort of linear method and weightings.

    So it basically comes down to B) which ones do you screen in and which do you screen out.
    and how do you justify your screening? and how do you characterize the uncertainty of your screening?

    B is always arguable.

  195. Steven Mosher says:

    “Well, yes, this does seem to be what is implied. It’s certainly possible that the issues are so great that we can’t actually even do the analysis, but this needs to be demonstrated and would suggest that not only do we know nothing about millenial temperatures, we can know nothing. I think most people regard attempts to understand something as preferable to claims that it isn’t possible”

    yes. On more than one occasion Steve and I have talked about our approach on temperature data. Lets characterize two approaches.

    A) Pour all the meat into the grinder and let your algorithms sort things out
    B) Select (by what criteria??) the most perfect data and work with that.

    I would say he is a Camp B kinda guy.

    Since I do actually get paid by folks outside of climate to make the A or B choice.. I can say
    it’s generally clear as mud and a choice that leads to rather heated issues–especially when
    folks have clear idea of how your data screening might swing the final answer.. Even total
    numbskulls get that if I apply really tight screening I can reduce the data so much that
    no conclusions can be drawn ( ie , sorry we need more perfect data )

    And yes.. the camp B guys tend to end up being the kind who suggest that we know less than we do.

  196. Steven Mosher says:

    ““and quietly sewed the patient back up.”

    quietly?”

    ########################

    Kill shot.

    you win.

    and you used a question to do it.

  197. Steven Mosher says:

    war

    ‘With a PR element to his professional work, ATTP should perhaps reflect on how this kerfuffle might be perceived by lay readers like myself who are genuinely interested in climate science but absent technical competence, rely heavily on behavioural indicators to assess credibility – much like a jury member does.”

    war,

    this is a slightly less sophisticated version of the Joshua approach to science.

    Let me give you an example. I know nothing about string theory.
    I read Muller and his thoughts about it.
    I read Lubos’s diatribe against Muller

    As a Jurist I would obviously side with my boss. duh. Lubos swore and was mean.

    So what have I done. Ive used the very things we disregard in science, to make a decision.

    here is the clue. the world is NOT waiting for me to take a position on String theory.
    Nothing requires me to decide which is right. If you ask me.. I dunno, i dont need to know
    I dont need to form a decision.

    Nothing requires you to make a decision about gergis or mcintyre. And you can be sure that
    if you use criteria that are foreign to science (Feynman was a skirt chaser) you are more subject to things like motivated reasoning.

    So here is the thing. Erase gergis. And you still have the question… how much will it warm?
    believe gergis.. and you still have the question.. how much?

    its a wheel that doesnt turn

  198. No need to make a decision. You can also just look at the evidence for both arguments, think about how it fits to the current understand and how this would need to be wrong in each case, try to come up with a way to test which of the two is right. And so on. In science you have to be able to endure ambiguity.

  199. Willard says:

    > Its not a reading. Its his position ( as expressed to me ) that much of the data is unuseable[.]

    Fair enough.

    That position still requires an argument that no signal could ever come out of the proxies. Unless we have an argument to that effect, nothing in the Auditor’s hostile takeover prevents him to present provisional and exploratory reconstructions except his own incredulity.

    In other words:

  200. Steven Mosher says:

    “No need to make a decision. You can also just look at the evidence for both arguments, think about how it fits to the current understand and how this would need to be wrong in each case, try to come up with a way to test which of the two is right. And so on. In science you have to be able to endure ambiguity.’

    yup.

  201. Steven Mosher says:

    “That position still requires an argument that no signal could ever come out of the proxies.”

    Err no it doesnt

    Take as an example

    You have a defined protocol for taking tree rings.
    if the metadata is not present to confirm that the protocol was followed, a purist could
    say, ‘nope’ aint gunna use that

    A pragmatic fellow would say

    1. I assume the protocol was followed even though I have no evidence that it was or wasnt followed.
    2. Subject to that assumption, here are my results.

    No argument about the signal is required. No position on the signal is required to make the
    argument that ‘Nope” aint gunna use that data.

    This is just basic stuff willard. Do it every day. And the minute you have a conclusion that is dependent on a data exclusion decision.. then the fun arguments start.

  202. Willard says:

    > No argument about the signal is required to make the argument that ‘Nope” aint gunna use that data.

    “Nope ain’t gunna use that data” is a position or a decision, not an argument.

    If you say “nope, ain’t gunna use that data, and nope, ain’t gunna tell you why,” then that position does not rest on an argument otters can check out for themselves. That’d be more haruspex than audit.

    An audit, as I understand it, would replicate (cf. Larry’s tweet) the study using the protocol the purist way. A real auditor would say something like: “while I have little faith is such reconstruction, here’s what it’d look like if it was done the way I’d like.” That’d be publisheable. In MM05a, we had a glimpse of that kind of work:

    One thing not often now mentioned is that in that paper, M&M actually did what many other critics should have done. They repeated the calculation with their criticism made good, to see what effect it had. This was in their Figure 1. They showed the effect of marking those four Gaspe years as missing, and then the effect of using a centered mean rather than Mann’s famous calibration mean. They got a surprisingly large difference, which has been much cited in recent days. This post reports on my investigation of that surprise.

    https://moyhu.blogspot.com/2014/03/mcintyre-mann-and-gaspe-cedars.html

    This kind of work may reduce otters’ gut feelings regarding the unusability of the data, even if “in passing” comments are added every paragraph or so.

    OTOH, if you want to argue that “nope that data is unuseable,” then you need an argument, preferably a formal one.

  203. “Nope ain’t gunna use that data” is a position or a decision, not an argument.”

    Nope its an argument, read the whole thing

    “You have a defined protocol for taking tree rings.
    if the metadata is not present to confirm that the protocol was followed, a purist could
    say, ‘nope’ aint gunna use that”

    You need to read harder. If…

    The argument is only use data that has evidence supporting that standard collection methods were used and documented.

    For chrissake that is part of the whole avoid strip bark argument and the Krumholz concerns.

    jeez.

    go back and read harder

    Everything, just google metadata and tree rings since you wont read everything.

    https://climateaudit.org/2009/09/26/briffas-yamal-crack-cocaine-for-paleoclimatologists/#comment-194548

    https://climateaudit.org/2007/03/19/up-to-date-black-spruce-ring-widths/#comment-82527

    But just to be clear. I think he would have to make a clear well documented case of why in every case the metadata was inadequate or the collection procedures were not well documented.
    I think we can agree on that.

    It may very well turn out that there is data that meets the exacting purists standards, and in practice he may be less of of purist than I think, but that’s my impression. My main point is that there is a defensible position ( only use the best data ) that would explain a refusal to do any reconstruction.

    I have little sympathy nowadays for this kind of position.

    That said, he doesnt owe anyone an argument about why he doesnt do your bidding.
    But its also fair to say that you cant be socially recognized in the field unless you actually try
    to a reconstruction.

  204. Willard says:

    > Nope its [“Nope ain’t gunna use that data”] an argument, read the whole thing […]

    The claim “Nope ain’t gunna use that data” may very well be a conclusion of an argument. That conclusion, to be reasoned, i.e. to be a part of a valid argument, should in principle follow from the audit being done : first the method part, then results, finally some conclusions.

    The argument could be of the form:

    (1) Here’s a protocol P with Data D.
    (2) I only vet protocols that haz metadata M.
    (3) P has no M.
    (4) Nope ain’t gunna use D

    From (1)-(3), I could sell (4).

    Now, compare (2) with what you said earlier, “much of the data is unuseable.” Notice the difference? “Much of the data is unuseable” ain’t following from these kinds of premises. It’s easy to see:

    (1) Here’s a protocol P with Data D.
    (2) I only vet protocols that haz metadata M.
    (3) D has no M.
    (4*) D is unusable.

    (4*) overgeneralizes (1)-(3). One does not simply dismiss data as unuseable when one shows willingness to only conclude that one won’t gunna use that data oneself. What’s missing is an impossibility claim that is more general than personal incredulity.

    From big conclusions comes greater argumentative responsibility.

    ***

    > But just to be clear.

    Do you even paragraph?

    So much needles in the eyes it is almost undecipherable.

  205. Eli Rabett says:

    “Suppose your proxy is at 165,2, 15.1.. Lower left corner”

    There are few enough proxys that you can use a grid where this does not happen.

    However, best would be to use geography to construct the grid, a kind of Baysean approach. To do it you would bring a geographer on board with a great GIS, tell her where the proxys are and nothing else.

  206. Eli Rabett says:

    Tolean pronoun policy of course

  207. dikranmarsupial says:

    “For some journals they do ask you for all the old reviewer comments and your replies”

    Rather easier now with the electronic reviewing systems that usually provide them for you. I used to keep folders of my old reviews just in case I needed them again. For many journals you routinely get to see the other reviews as well, which is good for improving the quality of your own reviews (at least I found that).

  208. angech says:

    Willard
    ” angech peddles in the same concerns 1U did before him, but with a Moshpitian syntax.”
    I love you.
    From the bottom of my heart.
    That is the nicest thing anyone has said about me ever, not the 1U part, the end bit.
    ATTP
    ” You redo the analysis and resubmit the paper which is now quite different to the original version. None of this means that it wasn’t something that appeared trivial that lead to all these changes.”
    Nor does it mean it was nefarious.
    The assumptions on both sides of the debate are quite black and white.
    The importance of Gergis’s work is what makes it matter so much to both sides.
    If this was a minor paper on Patagonian Toothfish (hope I am not offending anyone) I doubt there would be 3 drops of ink spent on it.
    As it is a major paper in the Climate debate it is going to get a lot of scrutiny.

    Anyone attacking the author personally is to be decried.
    Anyone attacking the science with scientific reasons is legitimate and should not be howled down either.

  209. dikranmarsupial says:

    angech, you appear to have missed my question “was the methodology in the new paper better or worse than in the old paper?”

    An answer to this question would be very helpful in understanding the nature of your objection.

  210. angech,
    I’m still not sure why you thought what one of the authors said about the older paper is somehow relevant to the newer one.

    As it is a major paper in the Climate debate it is going to get a lot of scrutiny.

    Maybe it’s major in the online climate debate, but I don’t that makes it objectively a major paper. It’s just a paper.

  211. Steven,

    A) Pour all the meat into the grinder and let your algorithms sort things out
    B) Select (by what criteria??) the most perfect data and work with that.

    I would say he is a Camp B kinda guy.

    I think it’s obviously perfectly fine for Steve M. to have a view. However, there are some things to bear in mind.

    a) If you only really express your view on your blog, it’s not likely to have much impact.

    b) If your view is essentially that the problem cannot be solved, your work is likely to have less impact that that of those who actively try to actually solve the problem (i.e., answer the question as to what the millenial temperature for Australasia actually is).

    c) If your narrative includes that the other parties are engaging in practices that border on being scientific misconduct (or it sounds like this is what you’re suggesting) your views may be dismissed by those who regard that as not an ideal way in which to interact.

    d) If you insist that you are completely correct and that the other party is completely wrong, then you might regarded as somewhat unjustifiably over-confident. I will add that I found it amusing to see people make comments about how the easiest person to fool was yourself, but who then appeared to apply this only to the other party.

  212. angech says:

    “Was the methodology in the new paper better or worse than in the old paper?

    It should always be better in a new paper but no, it was apparently worse.
    Problems were in consistency of application of predictive techniques.
    In looking for data that matched known temperature patterns, so they could use that data as a proxy record they matched some records for the corresponding year, some for the past year and some for the future year.
    When they found a match they used it.
    Hence 3 different ways proxies were selected, then combined.
    Which is not a good way to select data.
    You have 3 different years of ad hoc types of proxy mixed together.
    In Australia that is called a “Captain’s pick” in a derogatory way.
    When skeptics do it it is called cherry picking.

    Included is the second problem that choosing a match with data 1 year into the future makes the data predict the future.
    If the concept of a tree ring looking a year ahead at the temperature and deciding how much it should grow this year is a valid scientific concept I confess to being a little skeptical.

    It brings in that concept of wanting to find a correlation so hard that you will accept an idea that is unphysical so long as it fits.
    Like the stock market going up 50% in those years the the Green Bay Packers win the championship. ( not a real example)

    New paper methodology states
    “For predictor selection, both proxy climate and instrumental data were linearly detrended over 1931–90. As detailed in appendix A, only records that were significantly (p < 0.05) correlated with temperature variations in at least one grid cell within 500 km of the proxy’s location over the 1931–90 period were selected for further analysis-"

    The problem is they list 68 degrees of freedom which means they actually used 1920- 1990 de trending but claimed it was 1931-1990.
    The methodology did what it said but they have labelled the dates wrongly.
    This can be fixed by correcting the dates but then it will throw the trends out.

  213. Are you sure you’re not confused about this?

    The problem is they list 68 degrees of freedom which means they actually used 1920- 1990 de trending but claimed it was 1931-1990.

    I’ve just looked throught the SI. I don’t understand it fully, but it does seem that they used multiple screening methods, some of which involved the period 1921-1990, and others involved the period 1931-1990 with 1900-1930 used as verification.

  214. dikranmarsupial says:

    angech “It should always be better in a new paper but no, it was apparently worse.”

    Right. Do you think the methodology was worse according to the authors, the reviewers of the paper and the editor?

  215. dikranmarsupial says:

    The discussion seems to have diverged from angech’s original:

    “The problem therefore is partly on her reviewers, who all missed the simple single word change and passed it as fit for publishing. … Surely an issue in this age of peer review. … Gergis having said that she had detected this simple mistake should simply have corrected it.”

    The point is (as I mentioned earlier) there are two ways of solving the problem:

    (i) The quick fix – just change the wording so it accurately reflects what was actually done.

    (ii) The long road – go back and redo the experiment the way you meant to do it in the first place (possibly with some improvements reflecting the comments from the first round of reviews).

    Gergis chose option (ii), which is the right approach. Angech may think that (i) is the right thing to do, but Gergis didn’t and I agree. When you write papers the idea is to write the best paper you can, as your research reputation rests on the quality, not the quantity, of your papers.

    I agree with ATTP, if you have found genuine problems with the paper then write a comment paper for the journal and submit it for peer review and find out whether your analysis actually has some merit.

  216. I agree with ATTP, if you have found genuine problems with the paper then write a comment paper for the journal and submit it for peer review and find out whether your analysis actually has some merit.

    Ahh, I decided that my response was a little churlish, so deleted it. However, if it gets to the point where someone is very convinced that they have found genuine problems with a paper, submitting a comment paper is the normal way to go.

  217. dikranmarsupial says:

    Having written four/five comments papers (in one case there were two rounds of comments) I can confirm that it is the right way to go about it, although it can be a fair amount of work (and it meant I achieved a childhood ambition to write a paper about dinosaurs ;o).

  218. Willard says:

    > The importance of Gergis’s work is what makes it matter so much to both side.

    I believe it’s the other way around, angech: once Gergis turns into a classic ClimateBall episode, it becomes important. Your own arrow of causation is too sharply shooted to be plausible.

    I love you too, BTW. Now, please stop parroting the same thing over and over again.

  219. Magma says:

    submitting a comment paper is the normal way to go ATTP

    I did that once with a JGR paper in one of my fields. The authors were not appreciative, and wrote a reply that didn’t really address the critical issue I had raised. But their paper received few citations afterwards, and where it was cited my comment was usually noted as well. So the process worked as designed.

  220. dikranmarsupial says:

    “The authors were not appreciative”

    Of the four comments papers I wrote, none were effective in changing the minds of the authors of the original paper. This is not a climate thing, AFAICS it is also the case in machine learning and paleontology. It was somewhat of a disappointment to me that the paper refuting the residence time argument had no effect on the discussion on climate skeptic blogs, but it was worth a try.

  221. angech says:

    Willard
    Ok
    Happy to oblige. Clean exit.
    For you and the thread.
    ATTP has been more than generous to me and my views.
    Dikran, your defense earns recognition for supporting where your heart is.

  222. “Dikran, your defense earns recognition for supporting where your heart is.”

    “clean exit”? ;o)

  223. Hope this doesn’t the “clean exit”, but I thought I would post this figure from Gergis et als SI. It compares the main reconstruction (black) with one in which there was no screening and all 51 proxies were used (red dash) and one with no screening and using all the 36 proxies in the reconstruction domain (green dot). Doesn’t appear to be wild differences, but am not sure how the non-screening reonstructions would influence the 2SE.

  224. Willard says:

    “But unprecedented,” AT.

  225. Steven Mosher says:

    “There are few enough proxys that you can use a grid where this does not happen.

    However, best would be to use geography to construct the grid, a kind of Baysean approach. To do it you would bring a geographer on board with a great GIS, tell her where the proxys are and nothing else.”

    That is precisely what I am thinking about.

  226. miker613 says:

    ATTP, I really hope we’re sighing for the same reason.

  227. Pingback: 2016: A year in blogging | …and Then There's Physics

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s