I was just messing about with some model-observation comparisons and just thought I would post some of the results. I don’t claim that I’ve done these properly, so use with care. I will, however, explain what I did, so that it should be clear (and so that people can highlight any errors in what I’ve done). I went to KNMI Climate Explorer and downloaded the monthly tas data (near surface air temperature) for the CMIP5 RCP4.5 runs (selecting one member per model which produces 42 model outputs). Once I selected this, I produced two different outputs; one baselined to 1951-1980 (to compare with GISTemp) and one baselined to 1961-1990 and masked to be -70S to 80N (to compare with HadCRUT4).
I then went to the Met Office and downloaded the monthly HadCRUT4 data. I then went to GISTemp and downloaded the monthly mean global surface temperatures (which requires selecting this at the bottom of the page and then saving the ouput as plain text). I then simply plotted the surface temperature data over the model ouput and also plotted a multi-model mean. The resulting figures are below.
A few additional comments. I don’t know if I’ve done this correctly (these kind of comparisons are invariably a bit more complicated than it may at first seem) but I have tried to compare like with like. Although I have tried to take coverage bias into account when comparing the models with HadCRUT4, I haven’t used blended model output – I’m only using the near surface temperature from the models, while the temperature datasets are a combination of near surface temperatures and sea surface temperatures. I also haven’t tried to produce any kind of uncertainty interval for the models; I’ve simply plotted the monthly model outputs for all 42 models. Therefore, as I said above, if you do use these, use them with care.
...and Then There's Physics 
FWIW, your plots look quite similar to my own messing about.
Good to hear 🙂
ATTP,
You can use TAS and TOS (which is SST) at the same time, you just the weigth ocean and land temperature in a new series from GISS (say GISS-Adjust)
so i done this: (for 1950-2013)
First is to get the weighting, i use RCP4.5 TAS Mod Mean, then download TAS land RCP4.5 und TOS ocean RCP4.5. So then i weigth by the smallest quadratic error untill the Bias between RCP4.5 TAS Mod-Mean and TAS Land RCP4.5 + TOS Ocean RCP4.5 becomes to 2.75*10-^6K
Therefore i found the weigths:
Ocean: 0.741 (w1)
Land: 0.259 (w2)
Is written to:
RCP4.5 Adjust (t1) = RCP4.5 TOS (t1)*w2 + RCP4.5 TAS (t1)*w1
After this i make up a new Temperatur Series:(because GISS is extrapolating over the arctic ocean there we have to seperate ocean and land)
I use ERSST4v for the ocean and GISS-Land-only. So then i do:
GISS-new (t1)= ERSST4v (t1)* w1 + GISS-Land-only(t1) * w2
Next step was to Adjust the ENSO-Impact on the Temperature Series:
Which is done like this:
GISS-Adjust = GISS-new(t0)-(MEI(t-1)(Juli-Juni))*a
where as a= 0.07K
So baseline for GISS-Adjust and RCP is 1950-2013
So my try:
Greets
correcting: “so i done this: (for 1950-2013)”
Should be 1950-2015
PS: A year ago, i went in this discussion with Karsten Haunstein, i think its okay to post his view on this topic from a year ago: (no ENSO-Adjust)
He made up a Ice-Mask where he use TAS over sea ice to directly compare to Hadcrut4 by Cowtan and Way
Also shown the differences:
Seems that there is no divergence and the phases where is, it looks like ENSO is the cause
Christian,
When you created your blended series, did you have to download each model independently?
ATTP,
January 2016, no i just use the Mod-mean by 1 member per Model
Unless I’m missing something, there are 42 that have tas, but only 36 with tos, so I’m not sure how you determine which tas model to combine with which tos model.
ATTP,
To make simple:
modmean42 rcp45 tas
modmean38 rcp45 tos
I found 4x Models which are missing in TOS are negligible but if i would publish my results i would going to seperate them, but for blog posting i think its okay
Okay, I see what you’re suggesting. Thanks.
Anyway,
To goal is that, if you blending, it seem to be more correct then without. So to make clear, if you have much time for this, better try this then without. There is more you can do, since solar forcing is to high in RCP-Projection and Aerosols to low, you also can try to adjust for this by using some kind of EBM which you calibrate on CIMP5-Model.. or taking out El-Nino effects..
Under the line:
I think its more complicated to compare observation and models in a fair way and we (i do think so) let the people do this, which are working with or on it
Impressive, Takehome I get from this is, leave science to scientists. Simple when you know …
Never give a drummer a saxophone…
Skeptics think observations are going drop right out the bottom of the plume in a just a few years. I don’t think they understand how unlikely that is.
Pingback: Rassegna da week-end - Ocasapiens - Blog - Repubblica.it
“Impressive, Takehome I get from this is, leave science to scientists. Simple when you know …”
Actually that is not the take home I get from this.
With adequate description, or better a markdown document, you have a task that anyone
with skills in data analysis can perform.
What would be cool is for people to post their actual recipes
Yes, I agree. Having said that, these kind of things can be more difficult than it may at first seem and if one is pre-disposed to prefer a result showing that the comparison is poor, rather than good, it’s probably quite easy to do something that seems reasonable, that produces a mismatch, but that ignores some of the complexity. I think the Richardson et al. paper was a great illustration of this.
I should probably clarify the above as I’ve made it seem that Richardson et al, weren’t careful and found a mismatch. Richardson et al. showed that if you did a very careful comparison between models and observations (by ensuring you were really comparing like with like) then the models compared well with observations, despite many other claims that they did not.
I have this weird impression that quite a few scientists knew they were not doing a like-with-like comparison. Not because they wanted a mismatch, but either because they didn’t consider it would matter that much, or because they thought anyone would know the comparison would be limited because it was not a like-with-like. Richardson et al shows it isn’t all that trivial to do a proper like-with-like comparison, but hopefully any future papers that make a comparison like this will be asked to correct for the many known biases. It would definitely have made it harder for pseudoskeptics to use Fyfe et al (2013) as ammunition, if Fyfe et al had done the hard work Richardson et al did and then did the comparison. But then maybe Fyfe et al would never have been published, because “not statistically significant”…
“I have this weird impression that quite a few scientists knew they were not doing a like-with-like comparison. ”
I mentioned it a bunch of times, but giving out homework doesnt work… haha.
It is rather mind numbing work to do it right and until the pause using TAS was basically good enough.
Ideally I suppose we could construct a script so folks could just run it and get it done the right way.
I was also thinking of doing a Shiny App so that folks could do it interactively.. Something like WoodsForTrees but fetching model data instead.
When I finish my current project I’ll look at it again
Here ATTP watch this
Perhaps I should have said, leave specialty work to skilled specialists rather than science to scientists. SM, I don’t hold you harmless for your part in ClimateGate, where you enabled amateurs with an agenda, but you appear to me (I’m trying to be precise and specific here) to be interested in the truth, and the truth matters. The harm that was done is irrevocable, and may become more evident in the decades to come, unless you are quite elderly. Your part in that is between you and yourself.
I have minimal but not nonexistent expertise but maximal association with science, except with the guile and attack methods of fake skepticism, aka climate science denial, having been on the point of the arrow directly in many attacks. I had no resources but my own, and a conviction that the incredible is not credible. I use an “old lady kicking the tires” method, which is, if I have to, to look at the methods of selling rather than the hype that I can’t measure for myself, and to call in an expert when I need to know. I learned this from my mother, who learned it from yet another old woman.
What I got from aTTP’s effort was that a skilled “outsider” with ready access to the data could relatively easily produce results consistent with the conclusions drawn by the supposed “conspirators” indicating that there is in fact plenty of evidence, even if one is in a sealed room and hasn’t looked for oneself, that the climate is doing what it is doing. That we don’t meet someone’s “high” standards of evidence is irrelevant in a collapsing civilization.
For an example of what I’m on about, have a look at this question on Quora, which embodies dishonesty at the highest level. I do not apologize for talking about the way temperature is measured and how we dishonor science when we assume we know everything by looking at a thermometer and all other forms of temperature measurement are manipulated and capable of attack as “cheating”. Like all other forms of humanity, “honesty” is relative.
“SM, I don’t hold you harmless for your part in ClimateGate, where you enabled amateurs with an agenda, but you appear to me (I’m trying to be precise and specific here) to be interested in the truth, and the truth matters. The harm that was done is irrevocable, and may become more evident in the decades to come, unless you are quite elderly. Your part in that is between you and yourself.”
I dont hold myself harmless either. The current BatesGate or NothingBurgerGate is a good parallel.
A while back I had lunch with a fellow who shares you opinion and he asked me what I was doing.
I explained. From Nov 17 -19 I basically was answering a question put to me by Anthony. were these mails real. ( more explantion if you like ) on the 19th when they were made public I was done. Tom Fuller asked me what I thought and I said that professional journalists should handle it. I wrote Andrew Revkin a note, I told him to follow the FOIA, which was the only story worth telling.
A week later the news was dominated by two story lines:
A) Boys behaving badly
B) Fraud hoax etc.
On Nov 29th Tom wrote to me and asked me if I was happy with my decison to let the press tell the story and would i write a book with him telling the history and what we thought the real story was. basically, mails dont change the science, there was a wolf at the door, but FOIA is an important issue. So we wrote the book. From my perspective I knew that nobody would be happy. Skeptics wouldnt be happy with the last chapter and folks on the AGW side would not be happy with the sexed up quotes in the cover, I did many interviews with conservative press that ended up on the cutting room floor because I refused under questioning to call it a fraud. Most of them refused to understand my line “mails cant change science” They found my FOIA line to be boring and too nuanced.
a giant nothingburger.
The biggest lesson I learned is that you almost cant say anything critical about climate science without being used by contrarian extremists, And you cant say anything mildly supportive of skeptics ( I think Anthonys citizen project was great, I still look at the data they collected )
Funny story, I had lunch with an important silicon valley type. His question: is global warming real? My answer Yes. Good, he said, I wont buy your book. Do you think Machine Learning will take off? Yes.. Good, we need some ideas from the open source side of things, come in next week. For him FOIA was a big nothingburger. For me it was part of a whole drive for more openness and transparency.
if I had to do it over, I would have been more forceful about the fact that the core science was utterly unaffected by the mails and that, as I wrote in my submission to parliment, most if not all of the problems could be handled by improving some protocols about data and code.
The only solice have is that the investigations basically backed me up on the mishandling of FOIA. And its good to see more folks sharing data and code,
Finally, It puts me in a unique position to assess Judith’s actions. Like i said, been there done that. I would definately do it again, but I would A) take more time B) change the focus C) avoid the dog whistling and sexed up cover art.
Now if you ask Joshua Climategate had no proven effect, so my intentions were merely bad and it had no lasting effect.
I hope this doesnt change what you think of me, because what you think of me is none of my business.
Susan -SM may not be blameless, but his penance (actually reading and trying to dispel some of the nonsense at WUWT) is far worse than any priest required of me.
@attp and Christian:
This is the updated version of the blended comparison:
I used tos and tas from 23 CMIP5s (IIRC the one’s available at BADC, a mirror site for climate data) and treated them all separately as far as their ice extent is concerned (which is implicit in tos). I also like Zeke’s plot who did the monthly comparison just the way you did:
Nice Karsten..
Glad you wrote all that above Steven. Has helped me understand somewhat.
Cheers
Nathan
Mosh wrote:
“The biggest lesson I learned is that you almost cant say anything critical about climate science without being used by contrarian extremists, And you cant say anything mildly supportive of skeptics ( I think Anthonys citizen project was great, I still look at the data they collected )”
Doesn’t leave much room does it?
This may be true in the blogosphere, but amongst scientists, I suspect it is not. Scientists tend to be supportive of those who actually do something constructive, even if they have a rather poor reputation overall.
Karsten,
Very nice, thanks.
Steven –
=={ Now if you ask Joshua Climategate had no proven effect, so my intentions were merely bad and it had no lasting effect. }===
FWIW,
To clarify.
I don’t think that there was NO effect (thank you for qualifying your statement with “proven”). I don’t know what the effect was, and I question the many assertions by “skeptics” who say that they know what the effect was (usually by relying on the fallacious approach of extrapolating from their own anecdotal circumstances – a process of generalizing from unrepresentative sampling).
I think there was an effect in that something like Climategate gives people more ammunition to use in their identity-related struggles. It may have had the effect of making people more sure about their identity-correlated views, but I tend to doubt it.
The effect that I don’t think that it had, despite many claims otherwise – was to change public opinion on a meaningful scale w/r/t policy options to deal (or not deal) with climate change (i.e., a differential effect of moving people from one side of the great climate divide to the other, or as Judith like to claim, caused a “crisis” in the public’s faith in climate scientists).
The evidence I’ve seen is that a relatively small % of the public has much beyond the most surface level knowledge about Climategate. Of that small %, a relatively small number claim that it changed their views meaningfully. Of that small %, only a minuscule % moved from trusting climate scientists to distrusting climate scientists (that number is reduced by the % who claim to have gone in the opposite direction). Of the small % who claim that their views were changed, IO, the main effect of Climategate was one of confirmation bias. Therefore, among the relatively small % who (claim that) they lost trust in climate scientists (which means they’re saying it was there to begin with, which I doubt), the majority were libertarian, anti-government science, anti-mitigation policy types to begin with. And note that there is no realiable pre-test data which would validate the level of trust they had prior to Climategate so there’s no way to actually test any related conclusions.
Consider the current NOAAthingburger everyone in the climate-o-sphere is talking about. Has this event changed the views of a meaningful % of the public w/r/t public policy on climate change? I would guess not – although I wouldn’t find it entirely surprising if in some years hence, some people will claim that reading about NOAAburger caused the scales to fall from their eyes and convinced them that their trust in climate scientists was misplaced, and that indeed, climate scientists are tampering with evidence and cooking the books in order to fool the public and advance their global government agenda.
As for your intent, Steven, I assume that you thought that you were pursuing the “truth” about climate science. I would guess that the % involved here with some nefarious intent to spread lies or misinform is probably pretty small and I don’t see a reason to think you were in that category.
Steven Mosher, thank you. The uses and abuses of FOIA as a battering ram are causing a variety of problems. Out with the old, in with the new. In principle, it was a good idea, but it turns out the selective use of the result and the selective perception of people who look no further than affirming their prejudices will continue to be a problem (Clinton, Mike Mann, etc.). Email publication is so wholesale and vulnerable to selective quotation.
You’re right, the solution to our problems runs against ordinary people’s needs and appetites regardless of tactics, so we will likely fail. No doubt we will make it worse with forms of geoengineering that allow us to avoid the hard detailed work of changing our habits. Good intentions no guarantee.
It is always interesting to be challenged, and I like it. When I weigh in, if it is not just self-indulgence, I sometimes write first-hand about evaluating expertise without being able to follow the material, and think sometimes scientists don’t reach over the footlights in public communication with careful language about uncertainty and peer review (Dr. Holt in that hearing was a case in point; he spoke to the highest standard and was easily dismissed not in spite of but because of it). We all rely on people who know stuff we don’t, doctors, plumbers, engineers. Drawing the line between that and falsely assuming authority (e.g., Gaiever) is where it gets sticky. This is my father’s explanation for not weighing in on climate (aside from age and other interests), though he will be supportive if asked, and is quite clear about the issues.
Oops, note to Steven Mosher, not Joshua. Bad error … [Mod: corrected]
So now it’s all about FOIA and how scientists were harassed and goaded by it into arguably sub-optimal responses. And Steven Mosher is and always was nothing more than a fearless champion of the truth, openness and solid science. Good to have that cleared up. Hate a mess, I do.
Thanks for fixing that.
BBD, we cannot turn the clock back. I’m against circular firing squads when there are real enemies about. I don’t accuse you of being one, but for example the Berniebusters who are still agitating against pragmatists and centrists will keep Republicans in office, and that is the worst possible option. I call it the “purity monster” and had to get quite old before I realized how self-defeating demanding perfection from imperfect humans is, and how likely it is to enable to Bannons and Hitlers of this world. We need all the help we can get. We cannot inoculate ourselves against a large portion of the population.
ONeillsinWisconsin – https://andthentheresphysics.wordpress.com/2017/02/11/messing-about-with-model-obs-comparisons/#comment-91469 – speaks for me.
Meanwhile, this is all off topic. Good stuff from Karsten.
Susan A
Nor can we rewrite history. Steven’s artful self-exculpations always grate with me. But I agree with the rest of what you say. And indeed, off topic. ‘Nuff said.
RE the OP, I’ve also used the Climate Explorer CMIP5 data to create weighted TAS land + TOS model realizations. For all the RCP scenarios, I matched Climate Explorer’s “all models” runs for land-only TAS and TOS by model name; if the model name had at least one TAS run and one TOS, I created a weighted average (Earth is approx. 28% land, 72% water) of all the name-matched runs to crudely approximate the model data of Cowtan et al. 2015 [1]. The result is a set of these matched, weighted, global monthly runs with the following counts: RCP 2.6, 24; RCP 4.5, 34; RCP 6.0, 17; RCP 8.5, 31. Here’s an R data set for anyone who’s interested:
https://github.com/priscian/climeseries/blob/master/inst/extdata/cmip5/tas%20+%20tos/cmip5.RData?raw=true
… and here’s a plot of my TAS + TOS model runs along with HadCRUT4 Global:
It looks a lot like Figure 4b in Cowtan et al. 2015, but without their rigor. 🙂
[References]
1. Cowtan K, Hausfather Z, Hawkins E, Jacobs P, Mann ME, Miller SK, Steinman BA, Stolpe MB, and Way RG: Robust comparison of climate models with observations using blended land air and ocean sea surface temperatures. Geophysical Research Letters 42(15):6526–34, 2015. dx.doi.org/10.1002/2015GL064888.
[R Code]
## Intro to “climeseries” at https://github.com/priscian/climeseries
library(climeseries) # devtools::install_github(“priscian/climeseries”)
inst <- get_climate_data(download=FALSE, baseline=TRUE) # Must have data sets in working directory.
cmip5 <- get_models_data(ensemble="cmip5", subdir="tas + tos")
series <- c("HadCRUT4 Global")
plot_models_and_climate_data(inst, cmip5, series=series, scenario="RCP 8.5", start=1862, end=2020, ma=12, ma_i=12, baseline=1961:1990, center_fun="mean", smooth_envelope=FALSE, envelope_type="range", envelope_text="range", ylim=c(-1.0, 1.5), conf_int_i=FALSE, col_i_fun="topo.colors", col_i_fun…=list())