Amazing as it may seem, the whole tampering with temperature data conspiracy has managed to rear its ugly head once again. James Delingpole has a rather silly article that even Bishop Hill calls interesting (although, to be fair, I have a suspicion that in “skeptic” land, interesting sometimes means “I know this is complete bollocks, but I can’t bring myself to actually say so”). All of Delingpole’s evidence seems to come from “skeptic” bloggers, whose lack of understand of climate science seems – in my experience – to be only surpassed by their lack of understanding of the concept of censorship 😀 .
Since I haven’t written much about the adjustments to temperature data, I thought I might take this opportunity to do so. One reason I haven’t done so very often before, is that it’s not something I’m particularly familiar with, so I’m happy to be corrected by those who know more than I do (Victor Venema and Steven Mosher, for example). Let’s start, though, with a little thought experiment.
Imagine we want to create a dataset showing global temperatures over as long a timescale as possible; what would we do? Well, we’d design a temperature sensor and we’d place as many sensors as possible around the globe, in as regular a distribution as we could. We’d then take a temperature measurement at every location at exactly the same time every day. We’d also ensure that we didn’t move the sensors, change them in any way, change the site in any way, or change the time at which we took the measurement. If a sensor did need to be replaced, we’d ensure that the new one was calibrated to be exactly the same as the old one, and we’d keep meticulous records of everything related to the site and the measurements.
Having done this, we could then generate a record of temperatures for every site, from which we could determine a monthly average for every site. From this record, we could determine a long-term average for each month for each site (I think it’s actually for a region, rather than a site, but that’s not important for this thought experiment) and could then determine how the average temperature at each site for each month differed from this long-term average. This is called the temperature anomaly. We could then average all these anomalies to determine the global temperature anomaly. One reason for averaging anomalies, rather than averaging actual temperatures, is that it is less sensitive to missing data. Anomalies also allow you to better compare climatic trends at different locations that may have very different absolute temperatures.
So, it all sounds easy. The problem is, we didn’t do this and – since we don’t have a time machine – we can’t go back and do it again properly. What we have is data from different countries and regions, of different qualities, covering different time periods, and with different amounts of accompanying information. It’s all we have, and we can’t do anything about this. What one has to do is look at the data for each site and see if there’s anything that doesn’t look right. We don’t expect the typical/average temperature at a given location at a given time of day to suddenly change. There’s no climatic reason why this should happen. Therefore, we’d expect the temperature data for a particular site to be continuous. If there is some discontinuity, you need to consider what to do. Ideally you look through the records to see if something happened. Maybe the sensor was moved. Maybe it was changed. Maybe the time of observation changed. If so, you can be confident that this explains the discontinuity, and so you adjust the data to make it continuous.
What if there isn’t a full record, or you can’t find any reason why the data may have been influenced by something non-climatic? Do you just leave it as is? Well, no, that would be silly. We don’t know of any climatic influence that can suddenly cause typical temperatures at a given location to suddenly increase or decrease. It’s much more likely that something non-climatic has influenced the data and, hence, the sensible thing to do is to adjust it to make the data continuous.
So, once you have a continuous record for every site, you can determine your long-term averages, determine the temperature anomalies, and average these to get your global temperature record. There’s nothing suspicious about this, and it’s not some kind of major conspiracy; it’s basic data analysis. Also, for those who claim that these adjustments always increase the warming trend, as Richard Betts points out, one of the biggest corrections was the bucket correction, which actually reduced the trend.
Anyway, that’s my attempt to explain the reason for temperature adjustments. If anyone thinks I haven’t got something quite right, or wants to add to this, feel free to do so through the comments. To be fair to James Delingpole, he did link to a post by Zeke Hausfather that explains – in much more detail than I have – the need for temperature adjustments. If you want a more thorough description of why temperature adjustments are necessary, from someone who works for one of the teams that produces a global temperature dataset, it would be worth a read. You could also read some of Victor Venema’s posts, and you could look at the global temperature anomalies from four of the other groups that produce such datasets, and see if you can spot any major differences.
I’ll finish with Kevin Cowtan’s video that explains the need for temperature adjustments in the datasets from Paraguay, which is what started this whole kerfuffle all over again.
...and Then There's Physics 
Anders, Bishop Hill does not call Delingpole’s article interesting. Rather, he says of that article, plus several others repeating the same basic disinformation (plus two by Shollenberger on a different theme) that they are “All very interesting.” As I understand the convoluted shades of meaning of “interesting” in English usage, that amounts to “I want to endorse these claims without explicitly saying so”, rather than (as you interpret it) “Your claims are a load of complete bollocks, but I can’t find a polite way to tell you that.”
Tom,
You may well be right. I was trying to be flippant there 🙂 I was also thinking of Victor Venema’s post about how Judith Curry finds everything interesting (which I can’t quite find at the moment) which – IIRC – he interpreted as a bit of a catch phrase that allowed Judith to get away with posting almost anything. Actually, now that I think about it a bit more, you may have a point.
Many guess is that BH wanted to attack the authors of those articles because they are activists, but realized that if he did so he’d lose the argument, and so just said the articles are interesting instead.
A frequently heard cry is that, because surface temperatures have been homgenised, that there is some systematic manipulation of the data to fit the story of globally increasing temperatures. The same people who cry this (and mock/deny the need for adjustments to surface measurements) then go on to show satellite data, and espouse how much more trustworthy and accurate it is. As far as I know, what they fail to comprehend (willingly or not) is that the temperature data from satellites is – shock horror! – based on models, and require the same, or more, adjustments! And show the same trends!
The data I collect from my experiment requires a fair amount of adjustment before I can use it. If anyone asked me for the raw data, it might tell you something, but is largely meaningless and is pointless to be using it [without addition information].
Here are a few items I like to trot out when deniers*** resurrect their old temperature “data tampering” accusations.
The first is Proverbs 26:11 (it’s almost as if the Bible prophesied the rise of the climate-denial movement here).
The other items are plots of results I got when I tried my own hand at crunching the GHCN raw&adjusted data some time ago (using a simplistic procedure that I could teach to first-year programming students).
The first plot shows my raw vs. my adjusted vs. the official NASA results for the complete set of GHCN surface-temperature stations: https://www.dropbox.com/s/gmstiin2hp45iky/All-stations.png?dl=0
Note that all the temperature curves are pretty similar; however, my own raw temperature results (green) do show a small pre-1950 “warm bias” (and a smaller post-1960 “cool bias”) relative to my own adjusted temperature results (blue) and the official NASA results (red).
However, it turns out that many of the stations in the GHCN system have been moved one or more times in their history. The raw data, unlike the homogenized/adjusted data, does not incorporate corrections for those station moves.
One particular subset of GHCN stations that has been subjected to many station moves over the temperature record history is the subset of stations designated in the metadata as being located at airports. Many of those stations (particularly the “long-record” stations) did not begin their lives at airports, but were moved to airports (often from city centers) at some point during their history. Many of these stations were moved from city centers to airports during the mid 20th-Century.
So it is very instructive look at what you get when you filter out these “at airport” stations and reprocess the data without them. This next plot shows what you get when you do this: https://www.dropbox.com/s/f4yhbp7lqcl3naf/All-stations-not-at-airports.png?dl=0
Note how the pre-1950 warm bias in my raw results has been significantly reduced (ditto for the post-1960 cool bias). The match between my raw results and the NASA results (already pretty good) has been significantly improved with the elimination of those “airport” stations. This is entirely consistent with the elimination of a significant number of stations that had been moved from city centers to cooler outlying airport locations during the mid 20th-Century. (Declaring it proof would require much more extensive analysis).
Take-home points to throw at deniers*** are:
1) The data adjustments have a very modest impact on global temperature results. If scientists were looking to exaggerate the global-warming trend, they went to an awful lot of trouble to get very little additional warming.
2) Look at what happens when you eliminate a bunch of the stations that had been moved during their history — much of that bias disappears, as can be seen by comparing the two plots above.
This is a pretty simple story to tell non-technical folks. You don’t have to try to explain TOB corrections, or instrument-change calibration corrections, etc. “Station moves” is a term that anyone can understand, and “station moves” is perhaps the biggest factor that homogenization corrects for. Just show them the above two sets of results: the first where there no allowances for station moves, and the second where we’ve eliminated a significant number of stations that had been moved.
***I now use the term “deniers” without hesitation or apology.
caerbannog666,
I’m glad you commented. I was thinking of you when I wrote this. I remember you had an example of raw vs adjusted showing little difference, but couldn’t remember where it was.
Tom,
I realised that you may have misunderstood my interpretation of “interesting”. It wasn’t that they’re too polite to say that it’s bollocks, it’s that they couldn’t bring themselves to say that it’s bollocks, despite – deep down – knowing this to be the case.
The global temperature anomaly is largely due to sea-surface temperature (SST) readings. The fluctuating variability not due to the long-term trend of log(CO2) is caused largely by (1) ENSO, (2) volcanic disturbances, (3) expected TSI insolation changes, and (4) a multidecadal trend that follows LOD changes closely.
Something as trivial as Paraguay will not impact the variability at all and will barely nudge the trend if in fact a warming bias was applied due to “data manipulation”.
However, what will effect the trend is bucket corrections, as pointed out in the top-level blog post with reference to the Richard Betts comment. In particular, look at the bucket corrections that have to be applied during WWII. I mentioned this a couple of hours ago, via a comment to the previous blog post
https://andthentheresphysics.wordpress.com/2015/01/31/models-dont-over-estimate-warming/#comment-46050
Some of of us are way ahead of the curve 🙂
Just a quick followup. To show the effects of eliminating “airport” stations from the raw data results a bit more clearly, I uploaded two new images with just the raw data results vs. the NASA results (i.e. no “adjusted data” temperature curves). Eliminating the blue “adjusted data” temperature curve allows folks to see more clearly the changes in the raw data results (relative to the NASA results) when the “airport” stations are removed from the processing.
1) All stations: https://www.dropbox.com/s/cn2dafhbdbw3kxl/All_Stations_Raw.png?dl=0
2) Airport stations eliminated: https://www.dropbox.com/s/lk3kn4c6h7hecem/All_Stations_not_at_Airports_Raw.png?dl=0
I have now officially changed sides on the climate debate until all stations have a rectal thermometer. You don’t want to adjust one of those. Once is enough.
Man where do I start.
First some history. When I first had a look at adjustments done by NOAA for USHCN, all sorts
of red flags went up. They all looked to warm the record.
One could have three reactions to this:
1. Trust their work
2. Doubt their work
3. Check their work.
Options 1 and 2 didnt appeal to me much and 3 looked easy. So I dug in.
There were basically three types of adjustments.
1. Adjustments for a station moving. An example would be a station that was located
at 100 meters ASL being moved to 0 ASL. That station would cool on the order
of the lapse rate ( figure around 6C per km). This kind of station move would amount
to a cooling, a big cooling. A note for skeptics, even roy spencer has done this type
of adjustment.
2. Instrument changes; Anthony for example is big on the importance of instrument changes
http://climateaudit.org/2008/01/10/inside-the-ho83-hygrothermometer/
Well, when you look at the NOAA adjustments what do you find? Adjustments for changes
in instruments.
3. TOBS change. At first this puzzled me. But then I stumbled on some work done at a skeptical blog. AND, the work demostrated the need for this adjustment.
http://climateaudit.org/2007/09/24/tobs/
read the thread.
see my comment linking to this http://www.john-daly.com/tob/TOBSUM.HTM
At one point in the discussion mcintyre suggests this
http://climateaudit.org/2007/09/24/tobs/#comment-107771
So I did that. Zeke has also done that and hopefully will have a post soon.
The bottom line. TOBS bias is REAL and you need to adjust for it.
In every case I checked the following was true.
A) there was a bias associated with the change of observation practice. Skeptics above everyone else SHOULD GET THIS. In fact that is the point behind the very first work that Anthony did
with ‘changing paint’. Skeptics in fact have beat the drum really loudly about changes in observation practice being important. For example, moving thermometers close to buildings.
Changing urban enviroment. Moving thermometers off rooftops. They know these changes matter.. But of course they only focus on changes that potentially WARM the record in biased ways and tend to overlook the ways n which the record is cooled by observational changes.
B) the adjustments made were all defensible and the results could be duplicated. In one case ( TOBS) a skeptic had done the work. in another case ( changing altitude) another skeptic had
done his own lapse rate adjustment.
None of this convinced skeptics.
So there was another path to take. Compare adjusted to un adjusted
http://moyhu.blogspot.com.au/2012/02/ghcn-v3-homogeneity-adjustments-revised.html
The approach here is to simply point out that globally adjustments dont change answers very much. I’ve found the same in my own work prior to joining BEST. Others have found the same thing.
Faced with this answer skeptics will of course look at the extreme cases of those histograms.
With 40000 stations trust me we do have some extreme cases. And statistically I know that we probably have some of those cases wrong. That means that any skeptic out there can happen upon or look for a case that looks weird and then impugn the whole dataset or approach.
My email is full of them.
Hmm. I may get a chance to do a blog post on this. I have history on this topic going back to 2007. Or it might make sense to collect some materials that folks can use when fighting the misinformation on the topic. personally I think that is a better use of my time than chasing down
every false claim skeptics make about adjustments.
Since ATTP is trending……
and since the site is moderated….
it might be a good location to do a technical post.
Might take a while as we are working on some other stuff and I’ll talk to zeke about it.
ARRG
“That station would cool on the order
of the lapse rate ( figure around 6C per km). This kind of station move would amount
to a cooling, a big cooling. ”
Other way around.
Station moving from 0 ASL to 100 ASL is cooling.
You may well be right. I was trying to be flippant there 🙂 I was also thinking of Victor Venema’s post about how Judith Curry finds everything interesting (which I can’t quite find at the moment)
Strange you cannot find it, it has a very clear title: “Interesting what the interesting Judith Curry finds interesting“. 🙂
Victor,
I was being lazy, and cooking dinner, and was confident that you’d help me out 🙂
Someone was asking on Twitter if adjustments were made to other datasets. The figure below is the lightcurve for a planetary system that has 6 planets transiting/eclipsing the star. The top panel is the unadjusted data, the bottom is the adjusted data which shows the almost constant stellar brightness (look at the y-axis to see how small the variations are) with a whole series of dips which occur whenever one of planets passes in front of the star, blocking a small amount of the light (about 0.1 – 0.2 %).
Mosher, the Toms beat you too it.
Russell S. Vose, Claude N. Williams Jr., Thomas C. Peterson, Thomas R. Karl, and David R. Easterling. An evaluation of the time of observation bias adjustment in the U.S. Historical Climatology Network. J. Geophys. Res., VOL. 30, NO. 20, 2046, doi: 10.1029/2003GL018111, 2003.
Mosher demonstrates how the pseudo-skeptic blogs are very good at scoring Own Goals.
Thanks Eli. Those who do not want to read Russell et al. (2003) on the time of observation bias corrections, can find a short introduction to the problem on my blog.
Steven Mosher says: “there was a bias associated with the change of observation practice. Skeptics above everyone else SHOULD GET THIS. In fact that is the point behind the very first work that Anthony did with ‘changing paint’.”
By now I am very skeptical of any claim of a mitigation skeptic. I seems saver to assume that the opposite is true. Is there a source that confirms that white wash was used in the past to paint weather stations?
If there is, the experiment of Anthony Watts should be repeated by a real skeptic because that could be an interesting source of a bias that makes the trend in the raw data too small. If the paints were worse in the past and the sun could heat the weather screen more that would lead to higher temperatures in the past. Given that we have never seen any results from Anthony Watts experiment, it might be that he found an effect of the paints, but in the wrong direction.
Steven Mosher says: “Skeptics in fact have beat the drum really loudly about changes in observation practice being important. For example, moving thermometers close to buildings. Changing urban environment. Moving thermometers off rooftops. They know these changes matter.. But of course they only focus on changes that potentially WARM the record in biased ways and tend to overlook the ways n which the record is cooled by observational changes.”
Moving thermometers closer to buildings seems to be mainly an American problem. Explanations could be that America started early with automation, the cables were short and the technician had only one day to install the instruments.
A colleagues of mine from Mainz just showed that even in case of villages, the current well-sited locations are cooler than the ones used in the past. By half a degree.
Villages thus also have a little “urban heat island” and taking them out of it, produces a spurious cooling trend. What counts for the bias in the raw data is how well the siting was in the beginning and at the moment, not the “urbanization” itself.
==> “Villages thus also have a little “urban heat island””
As I recall, didn’t BEST compare the results from the rural weather stations to the urban weather stations, only to find that actually, the rural weather stations had warmed (slightly, but to a statistically significant degree) more than the urban ones?
Yes Eli I read the Toms
> since the site is moderated….
All climate blogs that I know are, e.g.:
http://moyhu.blogspot.com/2015/01/echo-chamber-at-climate-audit.html
Gavin Schmidt called the concerns about temperature homogenization nonsense.
If you want to see a counter example — one of data anti-homogenization, all you have to do is look at the global oil production numbers over time. The powers-that-be decided that to maintain the view that conventional crude oil production is increasing, they decided to add all sorts of liquid fuels to the accounting that aren’t actually conventional crude. It is very important to HIDE THE DECLINE in actual conventional crude oil production over the last few years. Can’t risk spooking the global economy.
So as an illustration of removing homogenized data, the classification of crude oil has expanded to include factors such as refinery gain and often even biofuels. This is what is referred to as “Total Liquids” by the international energy agencies that are “in charge” of the data.. Kind of like McDonalds decides to call their product a “Shake” and not a Milk Shake for obvious reasons.
I trust the temperature time series more than the oil production numbers, since the latter is in control of bureaucrats told what to do while the former is in control of scientists. The exceptions to this rule are government agencies that are held accountable, such as those from the UK and Norway. The data that they provide is disturbing as they show the real decline in crude oil production.
Kit Carruthers says: “The data I collect from my experiment requires a fair amount of adjustment before I can use it. If anyone asked me for the raw data, it might tell you something, but is largely meaningless and is pointless to be using it [without addition information].”
Do you have a pointer to the adjustments used in your field? ATTP already gave one. I would like to write a post about homogenization/segmentation/heteroskedasticity in other fields of science.
Also suggestions from others for such a post very much welcome. It is always hard to read the literature outside of your own field.
It is a growth industry in IT. Almost any dataset that involves humans recording data needs to be checked and potentially cleaned up before it can be used effectively whether for science or for marketing and data mining.
Google “data cleaning UK”. There are hundreds of IT companies dedicated to doing just that.
Is it really ‘amazing’ that there would be skepticism when decades old data is suddenly adjusted in a way that ultimately bolsters the arguments of the people doing the adjusting? Especially when some of the adjusting was done even though “you can’t find any reason why the data may have been influenced by something non-climatic”.
It may be scientific, but to the lay person, it smacks of being a self serving ploy, just so it can be claimed: “It’s worse than we thought”. I suggest climate scientists take a few courses in psychology along the way, to help them understand why they are often viewed with suspicion.
I know this might seem off topic, and I apologize to those who might find it distracting, but I just read a comment over at WUWT that, I think, ,really helps to explain the conspiracy the so-called temperature adjustments.
It takes place in an invaluable thread that helpfully traces the line of conspiracy through John Kerry, to John Holdren,, to Lord May, to Agenda 21, to Al Gore, to Paul Ehrlich, to Christine Steward, to Gina McCarty, to The Pope.
Here’s the comment:
General P. Malaise February 1, 2015 at 8:56 am
the Pope wants redistribution of wealth. which is very much against catholic doctrine since redistribution of wealth is socialist/progressive/communist speak for theft.
What else needs to be said? With arguments like that, and science websites like WUWT that provide posts with that much insight, is there any explanation that scientists could offer that would suffice as a counter argument? You just can’ argue against stuff like that.
The argument of the scientists is bolstered by having the best possible data.
You just put all the people you do not like into one green blob.
Even for Greenpeace it would make no sense to exaggerate global warming. There are more than enough environmental problems and global warming is very abstract and long term, there are better ways to increase membership if you are so cynical to think that that is their aim.
My last comment was a reply to Craig, not Joshua. Although Joshua’s comment fits well to the conspiracy theories of Craig.
I do not ascribe to conspiracy theories. They are the lazy man’s way of justifying preconceived beliefs that have no real evidence. I will leave that to the ‘birthers’. But it becomes very unsettling when science goes beyond the collection, analysis, and presenting of data, and becomes intimately wrapped up in the politics of the issues they are researching. Trying to influence public policy, regardless of how noble the cause, should be avoided.
I assume that Craig means that the scientists who established the links between smoking and disease should have avoided any interactions with public policy, thus leaving that to tobacco lobbyists, so he and any children/grandchildren could enjoy the benefits of starting smoking as teenagers…. Perhaps only people ignorant of the science or who do not like the results are allowed to have a voice in policy? (Sadly, in US, there is a fair-sized contingent of such, that would sadden Ben Franklin and Thomas Jefferson, who were both scientists, among other things.)
“Do you have a pointer to the adjustments used in your field? ATTP already gave one.”
Victor, my data is chemical analysis of water samples by ICP-MS (Inductively Coupled Plasma – Mass Spectroscopy). The raw data from the instrument is counts per second (CPS) of element masses. Meaningless in terms of concentrations on their own, other than to show relative amounts of each element for each sample, so the instrument requires calibration with standards of known concentrations. A straight-line regression through a series of standards provides the equation to convert CPS to concentrations. Since standards are prepared by manual dilution by the operator, occasionally a data point in the calibration sits way off the regression line. In some cases these are manually discarded to provide a better calibration. The calibration is also used to calculate the instrumental limit of detection (LOD).
The instrument measurements drift with time, so often an internal standard is run in every sample in a large batch and this is used to correct values later in the instrument software (a mystery to me, to be honest!). Finally, after the calibration and correcting for drift, we have a dataset. Values reported which are below the LOD are, in my case, converted to 0 value. Given the experiments I do, there is usually a set of blanks; the concentrations of these blanks (if > LOD) are subtracted from the concentrations of the other samples since this represents background concentrations superimposed on the experimental values. The dataset is then finally inspected manually for obvious outliers, which are removed from the data. An example of this is when we first started analysing for mercury, we found that it doesn’t wash out of the system very well between samples and so there is a residual signal in the next sample from the previous one. If the previous sample was a high concentration standard, for example, then you get a big (artificial) spike in concentrations with a systematic tailing off as more samples (and more washout) are run. These need to be identified manually and discarded as ‘bad data’.
Don’t have any graphs to hand, but hope that helps!
The cry of tampering was perhaps inevitable from the hardline den- rejectionist wing of the climate issue.
With Last year declared a record – even with the quibbles – the only response is to refuse to accept the data is accurate. So ‘implications’ are raised about the homogenisation process.
The problem with that argument is that it is just well-poisoning.
Ask WHICH past decade or years were really hotter than the records indicate and by how much… and tumbleweeds drift by.
Suggest ignoring the land surface data and just use the Sea surface data which has had a big adjustment DOWNWARDS…
No takers!
The smoking debate had one great advantage over the current climate crisis for proving its case: Millions of dead bodies, directly attributable to smoking. Unfortunately, that is what it often takes for public policy to catch up with science: Seat belts, air bags, child car seats, helmets, etc.
The public has a natural resistance against being told that their way of life is wrong, and needs to be changed. Climate science has not proved its case in the court of public opinion. It does not help when prominent politicians and scientists make sensational claims like the arctic will be ice free in 2015, or we will have devastating Atlantic hurricanes, sea level rise, etc. Because, when it turns out that we have a below normal hurricane season, and the global sea ice extent record is broken (albeit briefly), people will tune out, and the next time they hear even reasonable claims, they can just shake their heads, and roll their eyes .
Victor as you know the whole village question is very interesting to me.
( thanks for reading my blog)
Joshua
“As I recall, didn’t BEST compare the results from the rural weather stations to the urban weather stations, only to find that actually, the rural weather stations had warmed (slightly, but to a statistically significant degree) more than the urban ones?”
ya, I think even Peterson’s early paper had the same kind of answer.
The problem is that we still dont have a good “metric” or classifier for what counts as urban/rural.
Hmm, on of Oke’s students has a interesting new classification system, but classifying stations according to it is hard to do in an automated fashion. with 40K stations I need a good automated classifier.
I’ve probably spent 3 years on the problem. At one point Zeke and Nick Stokes and I did a AGU poster testing my super duper urban classifier. Worked pretty well. However, when I turned it over to berkeley for testing ( to replace the classifier they had ) my super duper classifier actually
didnt work so hot. Err it failed…. spectacularly. The answer came out even worse ( backwards) than the simple berkeley classifier.
Im still not happy with our Berkeley paper. And a while back started looking at village effects.
There are maybe a couple publications on it that are focused on 1 or two villages.
I was aiming at using GIS data and doing about 100 villages ( tiny towns) in the US.
Huge pile of work. I should probably go revisit it after readin the stuff that Victor posted.
==> “The problem is that we still dont have a good “metric” or classifier for what counts as urban/rural.”
Maybe.
But watching “skeptics” try to work backwards from that result, to try to invent some explanation that didn’t conflict with the conclusions they wanted to draw, was fun to watch.
Steven Mosher, just because you do not see difference between urban and rural stations or even see a smaller trend for urban stations does not mean that your method for determining whether stations are rural or urban is wrong.
Like for villages, what counts is how the quality of siting has changed. A measurement starting near the centre, which is nowadays outside of the city, maybe even on irrigated land or in watered gardens may well show an artificially low trend.
Craig,
Sorry, but this is essentially conspiracy ideation. You’re claiming that the scientists are being influenced by their political views. Where’s your evidence (that’s rhetorical). You shouldn’t really listen to all the nonsense being spouted on some other blogs.
WHT,
I find it quite ironic that, of all the major oil producing groups, OPEC appear to be better than the USA at providing data on crude, and only crude, production. None of that condensate nonsense, please.
Steven Mosher, to see how good your classification (urban/rural) is you may want to have a look at the difference between the raw data and the Berkeley estimate. If this difference has more of saw tooth character for the urban stations and more of a step function character for the rural ones, you are likely doing a good job. That would be a better indicators than the trend in itself. To compute this character (sawtooth/step) the two-phase or multiphase regression method of Lucie Vincent may be useful.
Forgot, another way would be to have a look at the number of breaks /cuts your find. A gradual inhomogeneity is seen by our cutting algorithm as a number of breaks and the relocations to the suburbs are partially additional breaks (some may align with observer changes which also happen in rural stations). Given that urbanization is just one non-climatic change of many, I would not expect to see too much of a signal in the variance of the difference time series, but the number of breaks may be more sensitive.
@- Craig
“The smoking debate had one great advantage over the current climate crisis for proving its case: Millions of dead bodies, directly attributable to smoking. ”
And yet people still smoke and the industry still exists.
Not least because of its past war on the science and ongoing resistance to anything that threatens its business model. Various ‘representatives’ are emerging at the moment opposing the introduction of plain packaging in the UK.
@-“The public has a natural resistance against being told that their way of life is wrong, and needs to be changed.”
Hmmm…
On the other hand a common trope of advertising is telling people that aspects of their way of life are inadequate, inefficient and wrong and if they need to change to this new and better thing…
And given the amount of change people have embraced over the last few decades, LPs-CDs-mp3s, landline to mobile, computer to phablet, to claim much resistance by the public seems unfounded.
Now the resistance raised by industrial and economic interests when told their profits conflict with public safety is another matter as seen by examples from tobacco to CFCs, Lead, asbestos…
@-“Climate science has not proved its case in the court of public opinion.”
Climate science has already proved its case in a much higher court (Nature), the lack of agreement in the ‘court of public opinion’ is irrelevant to the provenance of the science, but it does reflect on the failings of the ‘public court’.
Yeah Steve, you read the Toms, but did you believe them?
Victor, a long, long, long time ago, Eli looked at Willard Tony’s dirty pictures. There were as many that were going to have trouble with cooling as heating, and a lot of the heating ones were strange, like the big deal of someone throwing a trash burning can into the station enclosure. Who said they used it there. Lots of tree canopies shading the station, etc.
@ izen (love your new website btw!)
@ – On the other hand a common trope of advertising is telling people that aspects of their way of life are inadequate, inefficient and wrong and if they need to change to this new and better thing
And this is exactly where climate science fails miserably. Rather than trying to convincing people that they should change because it is a better thing, it uses scare tactics of global gloom and doom, none of which can be immediately demonstrated. Plus, what is being asked is not seen as a benefit, like a shiny new iPhone, but a demand for sacrifice. Time to get a new ad agency.
@ – And yet people still smoke and the industry still exists.
I rest my case.
@- Climate science has already proved its case in a much higher court (Nature)
Very emotionally satisfying, I’m sure. But I’m afraid it will have to be Nature itself that makes its case, one way or the other. It will be interesting to see who ends up with the bragging rights.
Craig,
No, this isn’t what climate science says. This is what a small group of mis-informers say that climate science says, so that another group of people can then say “climate science fails because it uses scare tactics” – as you’ve just done. Have you tried actually reading some climate science?
> Trying to influence public policy, regardless of how noble the cause, should be avoided.
When will you stop commenting, Craig?
@ Willard
Glad to oblige. Sorry to have burdened you with an outsiders view of things. Best of luck in your endeavors!
Although completely unadjusted data gives much the same answers on a global scale as properly corrected data, I think it’s worth removing as many confounding influences as possible.
As I noted on one of the WUWT threads complaining of ‘positive upwards adjustments’, complete with conspiracy theories and ridiculous claims that ‘warming in the USHCN is mainly an artifact of adjustments’:
“It could be argued that it’s better to look at raw temperature data than data with these various adjustments for known biases. It could also be argued that it’s worth not cleaning the dust and oil off the lenses of your telescope when looking at the stars. I consider these statements roughly equivalent, and (IMO) would have to disagree.”
Watts, not surprisingly, objected to this comment, although off the top of my head I can’t recall if this was before or after his (never-published) paper making huge claims from unadjusted data.
> Sorry to have burdened you with an outsiders view of things.
No problem. I don’t mind much if you stay, but please beware that the more you comment, the more you risk influencing public policy.
Thank you for your concerns.
@ Willard
Oh, I definitely plan on hanging around. I am a first class lurker, and love keeping up with all of the scientific blogs. Climate science is especially fascinating, although I admit I am more drawn to the social interactions than the science. AR5 was enough to convince me to leave the number crunching to others. But the complete polarization it has created among so many people is amazing! My other passion, Icelandic volcanoes, isn’t nearly as exciting.
Borrowing from Joshua and Willard.. Oh noes no grrrowth and starving children with less coal. The sky might fall..
> The sky might fall..
And then there’s James Bond:
Victor should take a look at a recent Nature paper challenging the primacy of evapotranspiration in urban-rural temperature contrast :
Strong contributions of local background climate to urban heat islands
Lei Zhao Xuhui Lee Ronald B. Smith Keith Oleson
Nature 511, 216–219 (10 July 2014) doi:10.1038/nature13462
The urban heat island (UHI), a common phenomenon in which surface temperatures are higher in urban areas than in surrounding rural areas, represents one of the most significant human-induced changes to Earth’s surface climate1, 2. Even though they are localized hotspots in the landscape, UHIs have a profound impact on the lives of urban residents, who comprise more than half of the world’s population3. A barrier to UHI mitigation is the lack of quantitative attribution of the various contributions to UHI intensity4 (expressed as the temperature difference between urban and rural areas, ΔT).
A common perception is that reduction in evaporative cooling in urban land is the dominant driver of ΔT (ref. 5). Here we use a climate model to show that, for cities across North America, geographic variations in daytime ΔT are largely explained by variations in the efficiency with which urban and rural areas convect heat to the lower atmosphere. If urban areas are aerodynamically smoother than surrounding rural areas, urban heat dissipation is relatively less efficient and urban warming occurs (and vice versa). This convection effect depends on the local background climate, increasing daytime ΔT by 3.0 ± 0.3 kelvin (mean and standard error) in humid climates but decreasing ΔT by 1.5 ± 0.2 kelvin in dry climates. In the humid eastern United States, there is evidence of higher ΔT in drier years.
These relationships imply that UHIs will exacerbate heatwave stress on human health in wet climates where high temperature effects are already compounded by high air humidity6, 7 and in drier years when positive temperature anomalies may be reinforced by a precipitation–temperature feedback8. Our results support albedo management as a viable means of reducing ΔT on large scales9, 10.
Mosher: There is no doubt that a TOB change can bias a record and that adjustments should be made. So what? Probably 99% of the empirical breakpoints being detected have nothing to do with a documented change in TOB. Peterson developed a systematic method for correcting changes in TOB, but it only applies to the US (a small fraction of the land surface). However, as best I can tell, no one currently uses the available metadata and correction methodology to correct US surface temperature data – they assume that empirical breakpoint correction algorithms will detect a change in TOB AND provide the correct adjustment. Has anyone ever compared empirical correction of TOB to Peterson’s systematic methodology?
BEST also creates a breakpoint-corrected record even though the mathematical process for analyzing the temperature field is somewhat different. Overtime a large number of breakpoints are present, the station trend matches the regional trend.
The real issue is that empirical breakpoint detection is finding more artifacts in station data than expected for TOB changes, station moves, and equipment changes. And the average breakpoint correction increases the trend modestly (20%?), without a good explanation.
If you look at a place like Tokyo (Berkeley ID #156164) where good evidence of UHI exists, you will see that the Berkeley breakpoint methodology is incapable of sensibly dealing with a gradually increasing bias. However, by assuming that discrete breakpoints are responsible for the bias, you still get what looks like a good fit to the regional composite. And the [improperly] adjusted Toyko trend differs by only 1% from the regional trend.
Frank,
I’d wondered about that. It makes sense. Given that you don’t expect discontinuities in the data, simply removing the discontinuities is really all you need to do. You don’t really need to definitively show that they’re associated with some explicit change.
Are there any examples of the Adjustments cooling the present i.e adjusting for UHI and increasing past temperatures ?
Any ?
jamesibbotson,
Yes, the bucket correction did exactly that. Remember, this is all relative to an arbitrary baseline, so an adjustment that warms the past, or one that cools the present, has the same effect.
With regards to land based thermometers affected by the well document UHI effect ? As i said ?
That link is for ocean based reading.
Jamesibbotson, Nick Stokes has a post just for you!
http://moyhu.blogspot.com/2015/02/google-maps-app-showing-ghcn-adjustments.html
jamesibbotson,
Okay, see Marco’s link.
Looking at Nick Stokes Map for the UK. It has two sites in Birmingham. Edgebaston and Birmingham / and some london ones.
Neither of which have had any adjustment applied.
Greenwhich in the MIDDLE of london has a POSITIVE adjustment on it. Even Kew gardens has a positive adjustment on it.
That can’t be justified. I’d like to see you try.
And not downward adjusting sites in major cities that have had huge growth over the past 20 years plus ?
jamesibbotson,
I’m not really interested in playing these silly games. There is, firstly, no physically plausible climatic reason for a discontinuity in a temperature dataset for a fixed site. It doesn’t make sense. Anyone who thinks it does is not thinking straight. Secondly, there are many reasons for adjustments. The station could move. The time of observation could change. The site itself could change (UHI, for example). The actual sensor could be changed. Your absolute statement this can’t be justified just illustrates that you’re not really thinking about this enough, and I have no great interest in discussing this with someone who has already made up their mind. If you think it’s fraud, or some kind of conspiracy, you’re welcome to think that. It’s a free world. I don’t have to waste my time convincing you otherwise and I don’t need to allow you to spread that kind of nonsense on my site.
Of course, if I’ve misjudged you, then feel free to prove me wrong.
In fact, according to BEST, both Kew Gardens and Greenwich were adjusted down.
+++There is, firstly, no physically plausible climatic reason for a discontinuity in a temperature dataset for a fixed site.\\”
Somebunny knawed down a bunch of trees or put up an apartment hutch. They started flying bigger jets. Who knows….
Frank says: “Probably 99% of the empirical breakpoints being detected have nothing to do with a documented change in TOB.”
In the USA many stations have a time of observation change. Around 1900 is was recommended to read the thermometers in the evening. Nowadays many observers also measure precipitation, the precipitation sums for the previous day are read in the morning. Thus many observers have shifted to the morning.
Before 1900 changes in the time of observation were quite common. Before the advent of national weather services there was not much standardization and many different methods were tried, measuring 2, 3 or 4 times a day, mixing such fixed hour measurements with minimum and maximum temperatures. And also the definition of the local time changed, especially with the introduction of rail roads the many different local time definitions switched to national time schemes.
Frank says: “Has anyone ever compared empirical correction of TOB to Peterson’s systematic methodology?”
Yes, that has been done. If you do not apply the TOB corrections, the Pairwise Homogenization Algorithm (PHA) of NOAA corrects these breaks and the trend over the USA is the same with and without TOB correction. Which is also evidence that the PHA works as it should.
Frank says: “And the average breakpoint correction increases the trend modestly (20%?), without a good explanation.”
In Global Historical Climate Network dataset version 3 (GHCNv3) the temperature change in the raw data is 0.6°C per century, after homogenization it is 0.8°C per century. We know from blind validation studies that the PHA algorithm used improves trend estimates, still it is nice to understand the changes.
Unfortunately not much people have studied such non-climatic changes hat make the trend smaller. Most studies on non-climatic changes are on the influence of urbanization. People wanted to be sure that the temperature trend was not due to a non-climatic change, thus studying urbanization was a priority.
What would you accept as a “good explanation”? Old measurements were often made in a way as to get more sun on the thermometer. Measurements were made at North walls (but the sun could still get on the instrument in summer during sun rise and sun down), on Stands in gardens, in half-open screens (Wild & French screen). Not only could the sun get onto the instruments, also heat radiation from the hot soil could get on the instruments as they were open to the bottom. These problems were solved with the introduction of Stevenson screens with double Louvre walls and a bottom.
The siting could have improved. Old measurements were often made in villages and cities, nowadays a large part of the measurements are outside of the villages and cities.
We have more irrigation nowadays and weather stations are important for agriculture and thus probably located where irrigation takes place. People water there gardens more, which could affect stations in (sub-)urban areas.
We have automatic weather stations nowadays. These sensors are smaller and thus receive less radiation. Some of them are additionally mechanically ventilated, which further reduces radiation errors.
jamesibbotson says: “Are there any examples of the Adjustments cooling the present i.e adjusting for UHI and increasing past temperatures ?”
There is a mountain network in the USA, which installed new automatic weather stations. There new ones were on average 1°C warmer. With the same Pairwise Homogenization Algorithm of NOAA these temperatures were adjusted down.
You may have missed that climatologists adjusted the data down. On WUWT it was presented as another indication that there is a warm bias. Whether you adjust up or down, the unreasonable will always complain.
(Sorry for messing up my italics in my last comment.)
Whether a station is in the middle of a city is not relevant for temperature trends. The UHI in itself gives a bias, you would need an increase in the UHI to get the wrong trend.
Even whether cities have grown enormously is not the relevant question. There is fresh air mixed in from above. The size of the urban heat island is thus determined by the surrounding of the station. Urban geographers estimate that the size of the relevant surrounding is about 500m. If this surrounding did not change there is no influence of the UHI on the trend. For London, Vienna and Sydney the trend in the city is not different from the one of surrounding rural stations.
In most cases the station also does not stay in the middle of a city, but is relocated (multiple times) away from the centre. Because the surrounding of the station is no longer good enough and because the meteorological office cannot afford the rents in the centre of a city. In this case, the relevant question is how strong the UHI was at the original location and how strong it is at the current location. It is thus well possible that urban stations have an artificially small trend.
Also when the station stayed at the same location, there can be recent cooling biases, for example the introduction of ventilated automatic weather stations or the watering of the gardens.
Both GISS and HadCRUT have adjustments to take care of the influence of urbanization. In both cases this does not change much, fitting to our current understanding that the net influence of urbanization on the temperature data is small.
@ Then there is physics….
Interesting you refereed to best when i explicitly stated ” Looking at Nick Stokes Map for the UK. It has two sites in Birmingham. Edgebaston and Birmingham / and some london ones”
Which shows the adjustments as positive. Based on the GHCN data.
0.34 for Kew Gardens.
In the middle of london
Adjusted up.
Birmingham and Edgbaston stations. Adjusted Up in GHCN.
DUMFRIES in scotland 0.41 positive.
An Adjustment of 0.41 is almost the temperature trend over the past 100 years.
Looking at the sites and results, GISS and NASS seem to have a predetermined “Trend line”.
Stations above or below trend are adjusted not for UHI, station moves etc to to make them fit the “trend”
Very odd. Data is what it is and shouldn’t be adjusted.
Whether a station is in the middle of a city is not relevant for temperature trends. <—— Really ?????
Best Tell NASA that as they have spent a fortune on a brand new Weather Stations for the USA based outside cities in remote areas explicitly so they aren't contaminated by UHI.
jamesibbotson,
Yes, I know. All I was pointing out was that BEST says they were adjusted down. Your initial intervention here was to ask if any stations have been adjusted down rather than up. The answer is yes.
No it is not. Data is not what it is and shouldn’t be adjusted. Data is adjusted all the time in all sorts of fields to accounts for factors that are independent/irrelevant for what you’re trying to determine. The adjustment to temperature datasets is to remove non-climatic influences. I’ll say this one more time. If you think a temperature dataset that is meant to represent temperature measurements at a fixed, unchanged site, taken at the same time each day, can have discontinuities, then you’re not thinking about this enough.
jamesibbotson,
Try reading what Victor actually wrote, not what you think he wrote.
Try reading what I actually wrote, not what you think I wrote.
And it is NOAA and not NASA.
And the trend in the US Climate Reference Network at pristine locations is larger than the one of the standard network which has urban stations. Maybe there is more than urbanization?
For giggles, it is absolutely true for just about any major city, NY, Paris, London, that population density is lower today than it was 80 to 100 years ago and more. In NY, when the weather station was moved to Central Park, the change was way downward, and you can certainly feel the difference in temperature as you move from Fifth Avenue into the park and up the hill where the tree shaded weather station is
Elis speculation is that the biggest UHI changes have been in suburban areas, most likely those that went from rural to suburban.
The CRN design was a backwards looking validation of whether homogenization methods worked. The short answer, as Victor points out, is they may overcorrect for the warming trend
http://rabett.blogspot.no/search?q=tom+Karl+USCRN
about Toms trick and experimental design
I would argue that there is a cooling bias in the raw data and homogenization can improve trend estimate, but is not perfect, it will undercorrect. Thus it makes sense that a reference network without problems with non-climatic changes produces a higher trend.
The error bars are hard to compute and likely still large, but you can at least say that there is no room for a large trend error due to urbanization in the USA and that it is more likely than not that there is actually a too small trend.
Its also worth pointing out that CRN will be more homogenous than HCN over the last few years in part because there has not been a long enough record to reliably detect breakpoints.
jamesibbotson,
There are plenty of large cities with negative adjustments. London, Paris, and Tokyo come to mind. Just use the search box on the Berkeley Earth website to find them: http://berkeleyearth.org/
Reno is often taken as the poster child for UHI bias: http://berkeleyearth.lbl.gov/stations/173102
As other folks have mentioned, however, you wouldn’t necessarily expect all urban stations to have a UHI trend bias. If the station was originally build in an urban area, the changing urban structure could have either cooling or warming (or no) biases. For example, a number of U.S. stations were on urban rooftops prior to 1940, before being moved to newly built airports or wastewater treatment plants. This causes a pretty big cooling bias.
jamesibbotson says:
February 3, 2015 at 12:12 pm
Are there any examples of the Adjustments cooling the present i.e adjusting for UHI and increasing past temperatures ?
Any ?
Here’s one (St Helena Island, WMO# 61901000)
Raw temps are plotted in green
Adjusted temps in blue
NASA/GISS “meterological stations index” is plotted in red.
https://drive.google.com/file/d/0B0pXYsr8qYS6eGptaUtpNDN2eTA/view?usp=sharing
Note the huge downward adjustment. Feel free to accuse scientists of trying
to mask the true global-warming trend.
jamesibbotson says:February 3, 2015 at 12:51 pm
“Looking at Nick Stokes Map for the UK. It has two sites in Birmingham. Edgebaston and Birmingham / and some london ones.
Neither of which have had any adjustment applied.”
The yellow means either no adjustment or less than 30 years data. Clicking will show the data period. You can use the Duration criterion to eliminate stations with less than 30 years data.
Nice Eli.
One project I have in the back of my mind is to look at urban areas like detroit that have been de populated ( going way back to some comments you made back in like 2008 or so– see I do remember the smart things you say)
The cool thing would be the infrastucture has largely remained intact while people just left.
hmm there are a few large cities in the midwest where this has happened…
trying to distangle the effects of various contributors to UHI..
More time in the day needed.. or maybe somebody has a grad student would like to steal the idea
ATTP wrote: I’d wondered about that. It makes sense. Given that you don’t expect discontinuities in the data, simply removing the discontinuities is really all you need to do. You don’t really need to definitively show that they’re associated with some explicit change.”
If you don’t know what caused the discontinuity, you have no scientific rational for correcting it. Suppose some discontinuities were caused by a gradual deterioration of station conditions, followed by maintenance that restored early recording conditions. In this case, correcting those discontinuities will introduce a bias into the record that wasn’t present earlier. Even a documented station move after gradual urbanization could restore earlier recording conditions. From my scientific perspective, each breakpoint correction appears to involve a untestable hypothesis that a sudden change to new recording conditions occurred, not a restoration of earlier conditions. (The same thing happens with BEST when they split the record at a breakpoint, preserving the biased trend and eliminating the restoration.)
Maintenance could involve cleaning the screen (higher albedo), improving ventilation, clearing ground vegetation, trees shading the station, etc.
Frank,
Of course you do. There is no physically plausible way in which a climatic condition alone could cause a discontinuity.
I don’t get the logic of this. What else would you do? You can’t just leave it as is as that would be clearly wrong. Sure, maybe there would be something non-discontinuous that could be non-climatic but at least you know that the discontinuities are non-climatic. You seem to be arguing for something extremely unlikely just to argue against correcting for something that is obviously non-climatic.
Zeke wrote: “Reno is often taken as the poster child for UHI bias: http://berkeleyearth.lbl.gov/stations/173102”
With 13 breakpoints in its record, one can always move the 14 pieces up and down so that they aline closely with the regional expectations. In this case the adjusted trend and regional trends are 0.70 and 0.75 degC per decade. However, UHI doesn’t occur at breakpoints, it develops gradually. So BEST isn’t making an adjustment that is appropriate for correcting a growing UHI bias.
The adjusted trend for any station appears to be determined by the “regional expection” developed from surrounding stations. If there are many discontinuities in the station record (as for Reno), the algorithm has no trouble aligning the multiple short segments on the “regional expectation”. If there aren’t many discontinuities (as for Tokyo), then the growing discrepancy between the local UHI-inflated trend and the regional trend APPEARS to convert a small natural fluctuation or small discontinuity into a very large breakpoint. In Toyko, the gradual bias introduced by increasing UHI appears obvious and is reflected in the size of the adjustments.
When you are looking for discontinuities by statistical techniques, you expect to produce some “false-positive breakpoints”. What is your expected false-positive rate?
It appears as if there is often a regular pattern of seasonal differences between a local station and a regional composite (higher in summer, lower in winter, for example). Is the magnitude of the seasonal differences taken into account when looking for breakpoints. Since 1970, Tokyo has oscillated (annually?) between +1 and -1 degC above the monthly regional expectation (monthly). After the stations are aligned, annual averages (13 month smooth?) are shown by BEST.
Frank,
No, I don’t you can guarantee this. All you’re doing is trying to remove the discontinuities. I don’t think this, alone, gives you the freedom to produce whatever trend you want. What they do is compare with regional trends afterwards to see if the resulting trend makes sense.
Maybe you can answer this. You seem to agree that there will be non-climatic influences in the temperature data. Do you really think we should be leaving these?
In statistical homogenization you compare your candidate station (e.g. Reno) with its surrounding stations. If there are no non-climatic changes, this difference is a constant and weather and instrumental noise. Any deviation from this, whether gradual or abrupt, is seen as a non-climatic change and removed. Thus also the gradual non-climatic changes are removed.
It is good to see that people now realise that this may not happen in BEST because they do use the surrounding stations to detect non-climatic changes, but I am not sure whether they do at the correction stage. When I told people these doubts at WUWT it was ignored, because I am an evil scientist and the erroneous claim in the above post was thus much more credible. Good to see that when no one knows any more that the caveat was revealed by a scientist, it is fit to be spread by the mitigation sceptics. Fits my world view.
In praxis it probably does not matter much, BEST finds about the same answer as GHCNv3 that does do the correction right using neighbours.
jamesibbotson says:
February 3, 2015 at 2:15 pm
Whether a station is in the middle of a city is not relevant for temperature trends. <—— Really ?????
If you think that UHI really is a major contributor to the global temperature trend seen in the surface temperature data, here’s your chance to verify that. Download the app at http://tinyurl.com/nasa-hansen5
Unpack and install/run per the included instructions (it’s not hard — if you can install Firefox + extensions, then you can do this). The app implements a very simplified (i.e. “dumbed down”) version of the NOAA anomaly gridding/averaging procedure. It uses the simplified algorithm to compute global-average results from stations that you select via mouse-clicks on a Google Map front-end. A station-selector control panel lets you filter stations by rural/urban status, data-record length, etc.
Raw and adjusted data results are plotted up against the official NASA “meteorological stations” index so that you can see how your own results stack up against the published NASA results. In addition, the number of your selected stations that actually report data for any given year is plotted up (i.e., if you select 100 stations in total, you can see how many of those stations actually reported data in say 1920, 1940, etc.).
The app is not a sophisticated analytical tool — it is a prototype “proof of concept” demo/educational tool that is intended to show folks how robust the NASA/NOAA global temperature results are. (I believe that skepticalscience folks are going to deploy a much more sophisticated web-based version in the near future).
Feel free to set it up, experiment with it — generate all the results you want with your own “hand-picked” rural stations and see what you get.
If you can demonstrate a real UHI effect (i.e. big differences between your rural vs. urban results), please post links to screenshots of your results here (easy to share via Google+).
Frank says: “When you are looking for discontinuities by statistical techniques, you expect to produce some “false-positive breakpoints”. What is your expected false-positive rate?”
The false alarm rate (POFD) is normally below 5%, the Pairwise Homogenization Algorithm of NOAA even below 1%. See Table 9 of Benchmarking homogenization algorithms for monthly data. The detection rate and the false alarm rate are, however, not very important for the question how accurate the trends can be estimated.
Victor Venema
“In praxis it probably does not matter much,”
I agree. I have just put up a post which compares TempLS based on unadjusted and adjusted GHCN. Even no adjustment doesn’t make a huge difference. Less than 0.05°C/Cen in the century trend.
Nice Nick.
We’ve finished some benchmarks that should interest people.
Frank
You can take all the data, throw out the stations with population greater tha 1 person per sq km
and you will get the same answer.
perfecting an adjustment for UHI is probably not possible since the phenomena is somewhat emphermeral
@ Steven Mosher
“perfecting an adjustment for UHI is probably not possible since the phenomena is somewhat emphermeral”
How about Roy Spencer’s idea of using population?
Credibility = Expertise + Trust.
While people making the datasets are undoubtedly expert it’s the trust that (generally) lets down people’s faith in the temperature reconstructions. I think people generally get the need for TOBS, and station move adjustments, but not why it generally always ends up with the past cooling and the present warming.
For example: How can it be that in 2007 acording to GISS the 1880 global air temperature anomaly was -0.12C, but that in 2015 the same anomaly was -0.43C (baselines for both are 1951-1980)?
It can’t be TOBS, or station move, as these are all in the past. New data? From 1880?
David,
New method?
@aTTP,
“New method?”
… and we are back to the trust issue..!
David,
The implied insult? Have you asked anyone? Maybe Victor Venema or Steven Mosher knows why the GISS anomalies have changed since 2007.
@aTTP,
“Have you asked anyone? ”
I am now :-D. I’d be interested to know. AFAIK versions 1,2 and 3 of the GISS dataset each seem to steadily cool the present and warm the past. As TOBS and station moves are ruled out (assuming of course they originally adjusted for them in V1), then as you say “new method” is the only factor left that could move the 1951-80 baseline, or recalculate the anomaly. What then was wrong with the V1 baseline? Why the need for the new method?
David,
Are you sure you meant it this way around? The normal conspiracy theory is that it always cools the past and warms the present 😀
David,
I’m not sure why I’m doing your work for you, but you could try reading this.
@aTTP,
“Are you sure you meant it this way around?”
@aTTP,
“I’m not sure why I’m doing your work for you, but you could try reading this.”
Thanks, but that seems to be a different issue, as it’s a) only from 2000 onwards, and b) only for the US., and c) “The effect on the global temperature record is invisible”.
Why do you have to lie like a rug?
WHT,
Maybe you could back that up. The data files DB links to seems to show the numbers he suggests. The figures I’ve found online, seem to not show this, though.
@WebHub,
Have you clicked the links?
I asked “Why do you have to lie like a rug?”
That’s a rhetorical question on my part. The reason you lie is that so you can inadvertently score #OwnGoals !
Thanks, as always.
OwnGoals in science are not always obvious like they are in sports.
The score differential keeps on growing and they haven’t a clue.
There is a bias in the raw data of the global mean temperature. In the raw data the trend is only 0.6°C per century and after removal of non-climatic effects the trend is 0.8°C per century (in GHCNv3).
Reasons for such a bias could be stronger radiation errors in the past and better siting, irrigation and watering in the present.
If there is such a bias, you would expect that when homogenization methods become better, they are able to remove more of the bias and the trend increases. Homogenization will become better due to better methods or due to having more data (in homogenization a candidate station is compared to a neighbouring station, if you have more data, they will on average be closer together and it is possible to see smaller non-climatic changes happening only in the candidate station).
For 1880 we do not have much data, thus in this case it is also easily possible that the value changed directly because more and more data is being digitized.
“How about Roy Spencer’s idea of using population?”
1. Roy used the wrong population data set.
2. Globally population is a weak predictor of UHI.
3. The effect of population varies with continent and wind direction.
4. it may be ok for MAX UHI, which is the thing that many people confuse with average UHI
Here is a paper that does a study of 419 large cities, looking at SUHI
http://www.ncbi.nlm.nih.gov/pubmed/22142232
I can probably dig up the whole text.
Here
Click to access peng-uhi-est-2012.pdf
see figure 4. population density is not a good predictor.
https://stevemosher.wordpress.com/2012/11/10/terrain-effects-on-suhi-estimates/
One way to view this is to understand that human beings put out about 100 watts.
That’s mousenuts in the energy balance equation.
HOWEVER, when you pave over grass and build a tall building to house those people,
the effects of the infrastructure required to house people can contribute to energy balance
via multiple pathways: change in albedo, radiative canyons, change in evaporation, change in surface roughness ( boundary layer effects).
So for example the building practices will matter more than the actual number of people.
Oke, The originator of using population to explain MAX UHI eventually abandoned the approach.
So, it doesnt make to much sense to try to build on an approach when the guy who came up with it tossed it in the garbage
In the ISTI benchmark (global validation dataset for homogenization methods) we would like to insert more gradual inhomogeneities in the urban stations. Any idea whether population density or city night lights is better for such a question? It does not have to be a perfect predictor, the gradual inhomogeneities themselves will also be stochastic, but it would be good if the relationship is similar all over the world.
“While people making the datasets are undoubtedly expert it’s the trust that (generally) lets down people’s faith in the temperature reconstructions. I think people generally get the need for TOBS, and station move adjustments, but not why it generally always ends up with the past cooling and the present warming.”
Simple.
I will speak to TOBS, but for the details you will have to wait for Zeke’s full treatment.
Historically the US network ( COOP) is a mess because it was made up of volunteers.
While other countries had standard observation times, the US did not. The observation times that the volunteers used tended to bias the measurement in one direction. That is just a historical fact.
Like the holocaust or landing on the moon. When you adjust for that the past is cooled.
Station moves are harder to generalize about.
Another way to look at this is by testing an algorithm BLINDLY.
you create a synthetic world that represents the “truth” calculate the average.
then have a separate team introduce biases into this world.
Then hand the algorithm team two worlds. one truth. the other biased. dont tell them which is which
See if the team’s algorithm can correct the biases.
Now if the algorithm was a secret warmist plot it would only make one type of correction.
Of course this test was done. not with 2 worlds but with 8. 1 true world and 7 biased ones.
What did we find? Algorithms move the biased world toward the truth. They do this regardless of their political agenda. hehe.
lastly, no one asks why our Algorithm does the opposite for Africa. That is the adjustments made for Africa are just the opposite of those made for the US.
I need to go fix that and make it warm EVERYTHING Bwaaah
“In the ISTI benchmark (global validation dataset for homogenization methods) we would like to insert more gradual inhomogeneities in the urban stations. Any idea whether population density or city night lights is better for such a question? It does not have to be a perfect predictor, the gradual inhomogeneities themselves will also be stochastic, but it would be good if the relationship is similar all over the world.”
Population density will probably be better. Nightlights has weird things in poor parts of the world
that are not electrified ( no lights, high population– think North Korea ) and also has oddities
like bright lights and no people ( think industrial facilities located in rural areas )
Hmm zeke and I did a whole study on these two. If I dont have a copy he does.
Impervious area and vegatative cover might be better.
Ideally you want a dataset that has a good spatial resolution ( 1 km or less ) and one that goes back in time. there are new global datasets of some of this stuff in final production
Stuff I have played with:
impervious area
population density
NVDI
nightlights
albedo.
urban land cover
“The implied insult? Have you asked anyone? Maybe Victor Venema or Steven Mosher knows why the GISS anomalies have changed since 2007.”
I stopped looking at GISS in detail some time ago, however, there approach has a couple things
that can lead to changing Anomalies month to month. This makes it look like they are re writing the past. They are not.
1. Changes in input data. NCDC is pretty diligent about cleaning up the root source files
GHCN-Daily. Daily, for the most part, feeds monthly (GHCN-M) and that feeds NASA.
When your provider changes your inputs… guess what? your estimate of the past will change.
2. Changes in reference stations. GISS uses RSM , or reference station method. One record
is selected as the reference station and long records are built by splicing shorter records together.
Algorithmically it is possible for the GISS reference station to change as more data comes in, since the base station is selected according to series length. if folks want a better RSM approach tamino built a kick ass one.
3. It is also possible for their UHI correction to change over time as more data comes in.
The general problem is that people think that GISS is producing an average. Hey, if I averaged
2007 records last month, why does it change when I re average?
Well, they are not averaging. They are estimating the temperature based on a statistical model.
That estimation will change if the past data changes ( NCDC upgrades) and that estimate will change because the algorithm uses all the information about stations to combine them
last example. We use the correlations between 1960 and 2010 to help estimate the fields prior to those dates. If I add 2000 stations that exist in that time period and if those additions change the correlation structure, then my estimate of the past ( say 1750 to 1960) may change.
It changes because
1. I assume the past ( 1760-1960) will have the same structure as 1960-2010
2. I estimate the past based on this assumption.
3. new data that is added to the present ( 1960-2010) could change the correlation structure
and consequently my estimation of the past.
Steve Mosher wrote: “You can take all the data, throw out the stations with population greater tha 1 person per sq km and you will get the same answer. perfecting an adjustment for UHI is probably not possible since the phenomena is somewhat emphermeral”
Since I can’t figure out what could possibly be wrong with the BEST analysis of the insignificant of UHI, I’m forced to accept the conclusion. I mention UHI ONLY because it is the clearest example of a bias that grows with time. When UHI bias is being corrected by at breakpoints (Tokyo is a clear example), it is clear that the correction methodology is inappropriate. There could be other biases that gradually creep into the record that are being handled inappropriately at breakpoints. Any breakpoint caused by a restoration of earlier measurement conditions (cleaning the station screen and increasing its albedo, vegation control, clearing ventilation, other maintenance) should not be corrected or split.
Consider this dilemma: Imagine two nearby sites that have the same temperature, site A and site B. The station is located at site A and a town grows up around it, GRADUALLY upwardly biasing the trend. When problems are recognized, the station is moved to site B, which has remained unchanged. When the resulting breakpoint is adjusted, the earlier temperatures at site A are lowered, even though sites A and B originally had identical temperature. In this case, correcting the breakpoint introduces bias into the trend. You need to correct a gradually increasing bias by adjusting the slope, not through a breakpoint. Unfortunately, it is far more challenging to detect and adjust a gradually increasing bias: You need to identify starting and ending dates for the bias and the needed change in slope.
You appear to be identifying far more discontinuities than expected for TOB changes, station moves and instrumentation changes. It is worth considering if all of these breakpoints should be corrected using or split when they might represent a return to original measurement conditions.
Frank: “Consider this dilemma: Imagine two nearby sites that have the same temperature, site A and site B. The station is located at site A and a town grows up around it, GRADUALLY upwardly biasing the trend. When problems are recognized, the station is moved to site B, which has remained unchanged. When the resulting breakpoint is adjusted, the earlier temperatures at site A are lowered, even though sites A and B originally had identical temperature. In this case, correcting the breakpoint introduces bias into the trend. “
Not only the breakpoint is adjusted, also the gradual inhomogeneity. You later seem to acknowledge indirectly this by writing: “You need to correct a gradually increasing bias by adjusting the slope, not through a breakpoint.”
Actually correcting for a gradual inhomogeneity by inserting multiple break inhomogeneities works very well in practice. If you take the weather noise into consideration it actually is normally almost impossible to chose between gradual inhomogeneity and multiple break inhomogeneities. Multiple breaks have the additional advantage that they are able to correct nonlinear gradual inhomogeneities and that there may also be break inhomogeneities during a gradual inhomogeneities.
But if you know of a scientific paper that demonstrates that there is a clear problem, do let me know. That would be interesting.
Given that the above post is about homogenization, I hope it is okay when I announce my last post on homogenization here: Changes in screen design leading to temperature trend biases. The second post in a series on the reasons why the temperature trend is too small in the raw data.
Victor,
Of course it’s okay, but I hope you mean “most recent”, not “last”.
🙂 Yes, is “last” wrong? Have been using that all my life. 😐 Thanks.
“Last” suggests there will not be any further posts. “latest” would be better, if you don’t like “most recent”.
Pingback: Guest post : Skeptics demand adjustments | …and Then There's Physics
Lest we forget, the weather in Parana has been a hot Rejectionist meme since it surfaced in Watts’ blog five years ago.
The whole issue can be scientifically settled by showing the raw vs. ‘homogenized’ data sets for all the stations used to derive the proof of global warming numbers. For each station, NASA would have to show records of when nostrum eats were moved, recalibrated to correct retroactivly for errors, changes in equipment, and so on, to validate alterations to the data.
They must certainly have detailed records of such changes to validate the adjustments they have made. Listing these would further settle the matter.
After seeing the recent Homewood / Booker articles, my curiosity was piqued. I have looked at the NASA raw vs. GISS ‘homogenized’ data for more than 100 stations I selected at random for the U.S. and Australia. So far, in all but one, and only very slightly, the GISS data turned a definite cooling trend into a clearly warming trend. It is hard to accept that a random selection would give that result. I cannot of course conclude that there is fraud by this small number of stations I have examined, but I will continue to look for the balance in reversals from cooling to warming and warming to cooling some have here asserted to exist. But so far, I feel like Diogenes looking fruitlessly for an honest man.
The denigration of anyone just taking a look at the data, and dismissing their observations because they are a ‘denier’ is unproductive, when disclosure of the relevant station by station data alteration specifics could so easily put an end to any speculation. Here is a golden opportunity for the ‘true believers’ to forever quash the ‘denier infidels’.
It is also notable that there have been no articles or postings identical to the Homewood RAW / ALTERED back and forth flash graphs showing other stations with such dramatic reversals from global warming to global cooling. It would be very effective in countering Homewood’s assertions, and proving, as asserted by several, that he cherry picked his stations, or that the true believers can just as easily cherry pick their own stations.
Drozier,
Firstly, it doesn’t have to be random. There can be systematic non-climatic changes that could produce an effect in one direction only. For example, changing the time of observation or changing the type of instrument used. Secondly, no one is denigrating those looking at the data. Looking at the data is a good thing. Doing so and the making accusations (implied or not) of fraud or misconduct is what is being criticised. The Booker and Delingpole articles clearly did this.
Something to bear in mind is that these observations are being used to determine climatic changes. Consequently, we wouldn’t expect to see any sudden changes in a the data series. Any sudden kind is much more likely to be non-climatic, than climatic and therefore it would be better to remove these changes than to leave them in simply because some people think raw data is somehow sacrosanct. It isn’t. Almost all areas of science adjust raw data isn some way in order to remove effects that aren’t relevant to what it is they’re trying to establish using this data.
“true believers”. With that you already know that drozier has made up his mind.
Well, drozier, as a “true believer” (in the scientific method), I can cherry pick one HUGE area of the globe that has been adjusted to cause “cooling” (i.e., less warming): the ocean.
http://variable-variability.blogspot.dk/2015/02/homogenization-adjustments-reduce-global-warming.html
There you go, temperature measurements of the largest part of the globe have been adjusted to reduce the warming. Those damn “true believers”, conspiring to reduce the warming trend!
Marco,
Yes, I had considered simply deleting Drozier’s comment, but thought it deserved a response. I don’t plan in participating in a lengthy debate with someone who uses “true believer” though.
Thanks for the responses. I guess I didn’t make clear that I only randomly selected station data looking for data sets that reversed a global warming trend into a global cooling trend to counter what Homewood suggests is prevalent throughout the global warming science community (for want of a better term).
I also understand that there will always be datapoint “flyers” that do not fit a particular data stream. In my experience, these are notes, but set aside for further analysis if there is no ascertainable cause.
The unanswered questions which invite asking are these:
1. Can the methodology in homogenization used by NASA or GISS be simply explained and specifically demonstrated, say, for just one of Homewood’s selected Paraguyan station’s data? For example, the Puerto Casado station?
2. For example, what instrument changes were made, and when?
3. What specific time-of-reading inconsistencies were found in the station logs, and when were they determined to have occurred?
4. Why, for example, we’re nearly all the early readings at Puerto Casado determined to be nearly 1.5 degrees erroneously higher than the homogenized calculations? And why are the current calculations equally erroneously low?
5. Has homogenization been continuously performed since the station was in place? If not, when was homogenization introduced?
6. Why not simply delete questionable temperature data rather than use values estimated or interpolated through homogenization? (If the actual readings for 80 years are so bad, requiring such large adjustments, why is this station even used?)
7. Since homogenization can so precisely better determine the actual temperature over the precision instrumentation presumably physically at the station, why haven’t corrective measures which would counter the need for homogenization been put into practice at the stations to yield better initial readings? (The adjustment of current readings suggest this question)
8. Which stations are used to determine the final temperature value that is used to pronounce whether we’ve had global warming or cooling for a year?
9. Of the stations that are used to determine the final temperature pronouncement of whether there has been global warming or cooling, how many are homogenized, and what effect as homogenization had on them?
10. Why are not all stations data homogenized? (Homewood states that he found 4 stations without homogenized data.)
Both “deniers” and “true believers” should want answers to these questions.
I’ve seen many comments by experts, such as Marco’s, that point to HUGE homogenized data sets which reduce global warming. But, I was asking to see station data sets that as equally and dramatically as the three Paraguyan station data, REVERSE a clear global cooling trend into a clear global warming trend at the specific stations, not which merely ameliorate the degree of warming or cooling. I have yet to see EVEN ONE flash superposed RAW / HOMOGENIZED graphical animation like Homewood’s. Such a presentation would dramatically discredit Homewood.
The only reason I have even looked at the NASA data sets is because I want to see for myself whether Homewood did indeed cherry pick his sites. The graphs he shows are truly in the NASA database. I verified them personally. I treat both “deniers” and ” true believers” with equal skepticism, especially the “deniers” when I see data alterations as dramatic as those that Homewood shows in his graphs.
I have been unable to find any data sets that mirror Homewood’s in the opposite direction, but I’ve only looked at about 100 stations. Right now, credibility is on Homewood’s side, but I intend to keep looking.
Finally, I put the terms “denier” and “true believer” in quotes to reflect what I have seen in most comments in responses, although not necessarily here, to the Homewood / Booker piece that precipitated this stream. Immediately, even referencing these terms has branded me here as closed minded, and not eligible to any longer post my thoughts.
Are my questions out of line, or not permissible?
Drozier,
I can’t answer all your questions, but here are some.
As I understand it, Puerto Casado has documented station moves that coincide with breaks in the data. So, if you see a break in the data and you know that the station moved at that time, a pretty reasonable interpretation would be that the move cause the break and that it isn’t climatic.
I don’t know that we actually do this. We’re more interested in long-term trends than year-to-year variability.
I presume that sometimes they don’t need it. If the observations have been at the same time each day, the instrument hasn’t changed (or a new one has been calibrated) and the site hasn’t moved or changed, then you wouldn’t need any adjustments.
I’ve found plenty. There is also the bucket correction for sea surface temperatures that significantly reduced the overall trend. Here’s an example from Iceland.
No, I think it really isn’t. Noone is disputing that adjustments take place. Homewood pointing out something that we all know is happening, without understanding why it’s happening, doesn’t make him more credible than those who are doing the actual analysis.
Drozier,
a) you did not put “true believers” in quotation marks in your first comment
b) there are few regions of the world which have seen any cooling trend, so that is the first problem with finding many examples of stations where a warming trend has been homogenized into a cooling trend.
c) you usually cannot get information on what exactly changed at a station at which time point and in what direction this will affect the temperature and by how much. The algorithms used by e.g. NOAA and GISS don’t actually need that information. Here’s one explanation:
and in more detail, also discussing the arctic issue:
Note e.g. that there are two known stations moves for Puerto Casado. That these caused artificial cooling is a conclusion from the homogenization routines – there is no other way of determining that those moves both caused cooling.
Also note that the homogenization introduces a warming bias compared to the raw data of about 10% over the whole record (more than 100 years). Quite unimpressive if the scientists are doing so much fraudulent manipulation…
d) Rasmus Benestad also has a discussion up on realclimate:
Also for that reason you should thus not expect to find many (if any) stations where a warming trend has been homogenized into a cooling trend.
Oops, sorry, ATTP for the embedding.
Oh, and ATTP, Drozier wants examples where a warming trend is homogenized into a cooling trend. As I explained above, those will be hard to find.
Marco,
Yes, I missed that and thought he meant a reduction in trend from raw to adjusted rather than an adjustment that turns a positive trend into a negative trend. Given that we are warming, finding a station where the adjustment changes a positive trend into a negative trend is going to be difficult.
Don’t know what happened with my long response, point d). The link to Rasmus Benestad’s article is here:
http://www.realclimate.org/index.php/archives/2015/02/noise-on-the-telegraph/
and that comment that followed that was supposed to refer to the T(obs) issue in the US and replacement of poorly protected weather stations with better designed systems. Both these issues cause artificial cooling.
Thanks for the answers, and for the video explanations.
It looks like the Berkeley graphs actually do show, with diamonds and squares, station moves and recalibrations respectively. So they must have actually reviewed station logs, something Homewood does not mention. Still, one would want to be able to study the justification for the direction and magnitude of the temperature adjustments. It is curious that not a single temperature measurement at Puerto Casado is the same before and after homogenization for any of the station locations or calibrations.
It would still be helpful to know which stations are or have been selected to determine the actual temperature for a particular year, and how they are selected. This would allow readers to see for themselves how the final temperature calculation was arrived at. It would also facilitate comparison with the stations omitted.
The second video shows that there is only a 10% or so difference in homogenized vs. raw temperatures, but what stations are actually used in making the determination?
Do they simply average all the selected stations final homogenized temperatures each year?
Still, a very great majority of the 115 randomly selected stations I have looked at thus far, about 95%, show a reversal of definite cooling to marked warming. A very few, two or three, show a slight warming before homogenization, with a greater increase in warming after adjustment. Several indicate no homogenization. Two show a very slight cooling, but so minor that replotting with lined base graphs with equal scaling would be needed for accurate comparison. I will try to get to the site shown in the second video to correlate the data adjustments to physical station changes.
The last question asked in the second video is, why would anyone bias the data, or something to that effect. If ones livelihood depended upon continuously increasing temperatures caused by the emission of CO2, and they knew the opposite to be true resulting in loss of their position, that would provide the incentive to do so. Similarly, if one knew there was indeed such a cause and effect, but whose livelihood depended upon emitting CO2, he would be inclined to falsify the data to that end.
Whatever is the case, falsehood from a source considered credible will unfortunately always trump truth from an average observer.
Perhaps I can help here. You may be familiar with John Christy and Roy Spencer, two prominent sceptical scientists at the University of Alabama, Huntsville (UAH) who for many years have curated a satellite-based atmospheric temperature reconstruction. Sceptics, remember. Not likely given to falsifying or unintentionally exaggerating a warming trend.
Better yet, the satellite data are *not* direct thermometry and they are not measuring surface temperature as are GISTEMP, HadCRUT, BEST, NOAA etc.
It’s no secret that a number of people are suspicious of the surface temperature data – especially GISTEMP, until fairly recently curated by James Hansen.
So what would a trend comparison between the commonly-used UAH ‘top lower troposphere’ (TLT) reconstruction, GISTEMP and HadCRUT show? I’ve put the three time-series on a common 1981 – 2010 baseline (native to UAH TLT) for ease of comparison:
UAH TLT vs GISTEMP, HadCRUT4; 1979 – present, annual means
The three trends are in very good agreement. The sceptical scientists and their satellite data are very close to the surface temperature records.
If there is a conspiracy to falsify the surface temperature data it is very poorly executed.
Drozier,
Most scientists are also University academics. This means that research is just one part of their job. They also have teaching and administrative roles and their jobs are very secure. If one particular hypothesis turned out to be false, they wouldn’t lose their jobs. There may be some scientists whose jobs have been specifically created to increase our understanding of climate change and who do not work in a University, which is why it’s always good to find out what the consensus position is. The consensus position is important since it’s unlikely that thousands of scientists from all over the world would be collaborating together to make it look as though the world was getting hotter.
By the same token, if you’re looking for a conflict of interest somewhere, then it’s very easy to see that a company which sells fossil fuels for a living is not going to easily accept that their livelihood is having a detrimental impact on our climate.
Drozier,
Maybe you should think about what Marco was pointing out. Given that we are warming, it’s not that surprising that adjustments do not produce cooling trends.
Lots of job depend on the existence of Reality. Reality depends on Grrrowth. Grrrowth ultimately biases everything toward Grrrowth.
Who can argue against more jobs, more Grrrowth, and more bias?
Thank you.
“It is curious that not a single temperature measurement at Puerto Casado is the same before and after homogenization for any of the station locations or calibrations”
Not sure I fully understand your comment, but I think you actually refer to something that is caused by the baseline period. NOAA uses the 20th century as baseline period, meaning that the summed temperature anomalies over this period equal zero. Suppose there is a step change upwards right in the middle of that period. Correction for that step change moves either the earlier half up, or the later half down. in either case the summed anomalies are no longer equal to zero. Thus, all anomalies must then be adjusted (downward or upward, respectively) to maintain this baseline period to give a summed anomaly of zero. Now not a single anomaly measurement is equal to before the homogenization.
“The second video shows that there is only a 10% or so difference in homogenized vs. raw temperatures, but what stations are actually used in making the determination?”
You can look it up yourself, just follow Kevin’s link and description in the video. Another option is to look through Nick Stokes’ many resources at http://moyhu.blogspot.com/
Do the red station dots on Kevin Cowtan’s video map represent those stations currently used to determine the global temperature?
How many dots/stations are there? ~40,000, ~7,000, ~16,000 (the numbers of stations totals I see in articles linked here)?
Do the dots show all the world’s temperature stations, or just a selected set of them?
Is there a link to his interactive map?
It is not possible for anyone independently check these data without knowing what stations are used. If not all stations are used, what determines which are used? My random selection opf stations around the world shows a very high percentage of complete reversal of global cooling to global warming or amplified warming after homogenization. (Cowtan’s shows in his video that homogenization increases the global temperature by about 10%).
My present approach is not scientific, so I’d be better of looking at the data sets actually used in the official determination of the global temperature.
Drozier,
I believe there are 40000 in total. If you go to the Berkeley Earth site, you can get the data for all of their stations and there is an interactive map.
You’re kind of missing the point of the scientific method. If you have all the data, you can develop your own independent algorithm. You can then compare what you get, with what others have got. That’s essentially what Berkeley Earth was set up to do. Guess what; it got essentially the same result as all the other groups had already determined. There are changes as newer methods are developed, but Berkeley Earth was intended to be indepedent and showed that there was no major issue with the global temperature datasets.
Drozier, yes, those red dots are the stations NOAA uses, as far as I know. It is the GHCN network.
Kevin’s interactive map is here:
http://www.ysbl.york.ac.uk/~cowtan/applets/ghcn/ghcn.html
You can enjoy yourself with Antarctica, for example, where there are quite a few that show warming in the unadjusted and then much less (to even cooling) in the adjusted. Remember also the area weighting.To take an extreme imaginary example: If you find fifty stations in Texas all adjusted upward from 1 to 3 degrees/century, and there are two stations in the whole of Alaska, but they have been revised downward from 2 to 1 degrees/century, the end result is hardly any change in the trend before and after adjustment.
Thanks for the link. This link shows the red station buttons, but does not include the interactive selection options below the map itself. If you can tell me how to get to the same map as in the video, that would help.
When comparisons to other stations are made, is homogenized data or raw data used? It seems any anomalies would be artificially exaggerated or reduced unless only raw data were used.
Are any physical checks ever made using calibrated reference instruments to verify that homogenizations are reasonable? For example, when the moves at Puerto Casado were made, were new measurements taken at both the old and new sites for comparison to the calculated data change?
I checked the Antarctic data, and did find a few before and after data sets that reverse raw data warming trends into cooling trends, but nothing as dramatic as the Paraguay upward shifts.
Drozier, I don’t think know what “interactive selection options” you refer to? Are you perhaps referring to this site:
http://www.skepticalscience.com/tempcalc.php
“When comparisons to other stations are made, is homogenized data or raw data used? It seems any anomalies would be artificially exaggerated or reduced unless only raw data were used.”
I’d say, check the actual articles, or the software. There are pointers in Kevin’s video (see also the information below the video) and here: http://www.ncdc.noaa.gov/ghcnm/v3.php.
“Are any physical checks ever made using calibrated reference instruments to verify that homogenizations are reasonable? For example, when the moves at Puerto Casado were made, were new measurements taken at both the old and new sites for comparison to the calculated data change?”
Regarding your second question, the answer is most likely no. Station moves are done by the local NMS, NOAA just collect the data and any other information provided by that NMS in the GHCN. I assume there are occasional comparisons when an NMS indeed does such comparisons, but I think Victor Venema knows a lot more about that (ask him at http://variable-variability.blogspot.com/).
“I checked the Antarctic data, and did find a few before and after data sets that reverse raw data warming trends into cooling trends, but nothing as dramatic as the Paraguay upward shifts”
It was just one example. I gave. Also note the area that this will affect. The Paraguay upward adjustment affects an area that is much smaller than e.g. the downward adjustment of Amundsen Scott. Finally, remember the difference between land-only and land+ocean.
Nick Stokes has some discussion on this, too:
http://moyhu.blogspot.com/2015/02/breakdown-of-effects-of-ghcn-adjustments.html
and a useful interactive map here:
http://moyhu.blogspot.com/2015/02/google-maps-and-ghcn-adjustments.html
(did not work for me on IE, but worked fine with Chrome). You can color the stations by adjustment – up, down, or essentially no change.
Below the station map in Kevin Cowtan’s video, there is a panel for selecting ‘Action’, to select or remove ‘By’ station, region, country, or latitude band, and ‘Or’ select all and clear all. He shows the cursor moving and selecting these options.
This panel is not below the map you linked to me earlier, only a Select Station message. Maybe someone knows the other link, or I have not properly gotten to it.
The NOAA adjustments look not unreasonable as Mr Cowtan explains them. I’d have to review the methodology to fully understand them.
Since these adjustments have the effect of raising the global land temperature by 10% over the raw data, they have at least the appearance, even without the intent, of a possible purposeful manipulation to artificially exaggerate the global warming trend. Using adjusted temperatures only to determine global temperature gives talking points to those who have a degree of skepticism, particularly with the recent NOAA pronouncement that 2014 was the hottest year ever by 0.04°C, well within adjustment range of the land temperature data alone.
Could not a minor selective change in which stations are used to determine the global temperature from year to year produce a 0.04°C difference in the calculated temperature? We need to know exactly which stations are or were used each year to make the final temperature determinations, something not made readily apparent by NOAA/GISS/Berkeley et al.
At the very least, the temperature proclamations should be openly presented in both raw and adjusted numbers, and also the results using all stations and not only the selected stations. As it is, the appearance to some may wiill be that the ‘fox is guarding the henhouse’.
Earlier, above, the issue of “why would anyone engage in such shenanigans” of tampering falsely with the data came up. Looking into the “follow the money involved” cliché with respect to carbon taxes, carbon credit brokering, and fossil fuel subsidies gives additional very plausible incentives on both sides of the global warming issue. Potentially many trillions of dollars are in question. Who in the global warming carbon tax/credit proponent community owns shares in credit trading brokerages (other than Al Gore)? Who in the skeptic community owns shares in the fossil fuel side (other than Big Oil)?
I suggest you send Kevin Cowtan an e-mail, I am sure he can help.
The rest of your comment…well, this is slowly moving towards conspiracy lala-land.
“Since these adjustments have the effect of raising the global land temperature by 10% over the raw data, they have at least the appearance, even without the intent, of a possible purposeful manipulation to artificially exaggerate the global warming trend.”
Someone who thinks this is completely impervious to facts, so in however much detail scientists explain things, it won’t change any minds. And again, the adjustment to the SST is reducing the trend – surely those bad, bad scientists would come with a way to adjust the SST to increase the trend, not decrease it…
“At the very least, the temperature proclamations should be openly presented in both raw and adjusted numbers, and also the results using all stations and not only the selected stations.”
You want the scientists to report data that is known to be flawed (the raw data)?!
Besides that, the scientists already use all station data that is available in the GHCN.
Regarding carbon tax and Al Gore: Gore actually proposed a revenue-neutral carbon tax in the early days. But in the US the word “tax” is so toxic that he switched over to the credit system, which supposedly is more “free market”.
If you don’t already know about it, you might be interested in this Blog, written by Victor Venema, who is working in the field
“At the moment most of my work and this blog is about the removal of non-climatic changes (variability) from historical stations data, which is called homogenization. I am probably best known for having been granted to be the first author the validation (benchmarking) study of statistical homogenization methods of the COST Action (large European project) HOME.
Because of this study, I am now in the Benchmarking and Assessment Working Group (BAWG) of the International Surface Temperature Initiative (ISTI). Within the ISTI I am also leading the Parallel Observations Science Team (POST), that aims to study non-climatic changes in (daily) climate data using parallel measurements representing the old and new situation (in terms of e.g. instruments, location).”
http://variable-variability.blogspot.com/p/about.html
Five statistically interesting problems in homogenization. Part 1. The inhomogeneous reference problem
http://variable-variability.blogspot.com/2013/07/five-statistically-interesting-problems.html
Chris Shaker
Chris,
Thanks. Victor is a regular commenter and I linked to a number of his posts at the end of my post 🙂
Have any of you guys been through the computer code that is being used to Homogenize temperature data?
If the code is written in FORTRAN, I would love to go through it just to convince myself there are no bugs in the code or method. A few checks like running backwards through the data would be on my audit list.
A standard method for finding hot-spots on chromosomes always gave highly significant results at the far end of the chromosome – thus the important end swapped when the direction of analysis was reversed. (FYI the hot-spots were locations where there appeared to be a gene associated with a measurement like Milk production per cow per year).
Do any of you actually have access to the computer code?
Do you know if simulated test-data-sets have been designed to check that the software can detect the simulated effects like instrument replacement?
Sandy,
Well, Victor Venema does this kind of stuff professionally, so you could always ask him. Nick Stokes has done some of this stuff, so you could always check with him too. There is a link to a code that I once used, but I have to rush off now, so will try and post it later.
“What we have is data from different countries and regions, of different qualities, covering different time periods, and with different amounts of accompanying information. It’s all we have, and we can’t do anything about this.”
Yes we could.
Claims of definitive fact could be knocked on the head and the resultant trillions saved indulging this guesswork could be spent creating a meaningful climate study infrastructure.
The idea that average temps for the entire planet can be measured to tenths of a degree (nay, hundredths in the hottest ever 2014 claim) under the present Heath Robinson set up is farcical.
Barry,
Are you telling me you have a time machine?
Technically, this isn’t what is being claimed. Am I right in suspecting that you’re not really that interested in learning more?