In my post about Nic Lewis’s latest estimates, I suggested that if his prior produces a posterior distribution that indicates that the ECS is more likely less than 2^{o}C, than above 2^{o}C, maybe he should consider a more physically motivated prior than what he currently prefers. I also suggested in the comments that

One might argue that if Nic Lewis is going to continue publishing papers suggesting a non-negligible chance of an ECS close to – or below – 1K, then he will have to provide some kind of physical motivation for such a scenario.

Of course, I didn’t define *non-negligible*, but some of Nic’s results suggest an ~ 17% chance of the ECS lying below 1^{o}C.

To be clear, I think Nic’s work is interesting and that this work makes a positive contribution to our understanding of climate sensitivity. My basic point, though, is that if you are investigating some kind of physical system, and your analysis produces results that are somewhat inconsistent with our physical understanding of that system, then you – or someone else – will eventually have to provide some kind of physically-motivated justification for those results. Nic did respond to this point, by saying

OK. The fact that my study’s results show a material amount of probability at ECS levels of below 1 K, whilst some other evidence suggests that is unlikely, seems irrelevant to me. The normal scientific methods involves individual studies reporting estimates based on their own data and analysis. …..

As for paleoclimate studies, some of them show significant probability at low ECS levels. Eg, Hargeaves et al (2012) GRL had a similar 5% uncertainty bound to that in my paper.

I think I know what Nic is getting at in the first part of his response; his work stands by itself, so he’s not obliged to explain this inconsistency. In a sense, this is true, but it doesn’t make the inconsistency *irrelevant*. At some point we will want to understand why different methods give somewhat different results.

^{o}C, and a median of 2.5

^{o}C. I guess a 5% probability of an ECS below 1

^{o}C is non-negligible, but it’s quite a bit smaller than 17%, and their result still suggests that the ECS is more likely above 2

^{o}C, than below. There’s also a more recent paper (Using palaeo-climate comparisons to constrain future projections in CMIP5, by Schmidt et al. 2014) that seems to update this slightly. Their result produces a 90% range of 1.7 – 4.9

^{o}C and a median of 3.3

^{o}C; somewhat higher than that in Hargreaves et al. (2012).

**[Edit : As Paul S points out in the comments, this is the prior. The result is a 90% range of 1.5 – 4.7**

^{o}C and a median of 3^{o}C.]So, it’s a little odd that in response to my point that the physical evidence suggests that the ECS is more likely above 2^{o}C than below 2^{o}C, and probably not below 1^{o}C, Nic refers to a paper that essentially suggests exactly this. I will add, though, that both Hargreaves et al. (2012) and Schmidt et al. (2014) might be biased because the models don’t consider atmospheric dust or vegetation changes. Specifically, Hargreaves et al. (2012) say

The PMIP2 experimental protocol for the LGM omits forcing due to atmospheric dust [Claquin et al., 2003] and vegetation changes [Crucifix and Hewitt, 2005], but while these are poorly constrained, they are likely to be net cooling influences. …. We can attempt to account for this (at least approximately) by increasing the modelled tropical cooling results of the PMIP2 models by a factor of 1/0.85. When we do this, the median Bayesian posterior estimate is reduced to 2.0

^{o}C with a 5–95% range of 0.2–4.^{o}C, and the regression-based estimate falls to 2.0^{o}C with a range of 0.8–3.6^{o}C.

So, atmospheric dust/vegetation changes could reduce the ECS range and might reduce the median estimate to 2^{o}C. Of course, the more recent Schmidt et al. (2014) paper suggests a higher median and slightly higher range than Hargreaves et al. (2012), so this effect may not be quite as significant as Hargreaves et al. (2012) suggest. To be honest, I’m not entirely sure I get this, but I think it’s because the vegetation changes and atmospheric dust are feedbacks (not forcings) and so they influence the temperature, without changing the forcings.

Anyway, my basic point was that the physical evidence appears to suggest that the ECS is more likely above 2^{o}C, than below 2^{o}C. Most paleo estimates appear to suggest the same. If Nic Lewis is going to continue presenting results that suggest it is more likely below 2^{o}C than above, then I think it would be in his interest to try and present a physically motivated argument in support of this. As Dikran Marsupial (who happens to be a statistician) says

the hierarchy is physics > statistics >= chimps pulling numbers from a bucket

I’ll end with that 😀

As i understand it Nic Lewis’s estimates are based on measurements that go back no more than 150 years at the most (and in some cases, such as deep ocean heating, only about 20 years). So his estimates of ECS can only take account of climate feedbacks that are already active in the system. Since other threshold-cued feedbacks have not kicked in, doesn’t this make it inevitable that his estimates are too low right across the board?

paul,

In a sense, yes, and – to be fair to Nic – he does discuss this in his recent paper and does clarify that his ECS is really an Effective Climate Sensitivity, rather than a true Equilibrium Climate Sensitivity. There are a number of related factors. The globally average feedbacks might increase with time as we warm. This Realclimate post discusses this. However, as Isaac Held mentions this could simply be because of regional variability (polar amplification) rather than because the feedbacks are actually non-linear. Then there is the possibility of the feedbacks actually being non-linear. Then there’s the issue of slow-feedbacks. Technically, the ECS is the fast feedback response only, and doesn’t consider the impact of slow feedbacks. Overall, you would probably expect these other factors to increase the ECS so, yes, Nic’s estimate is probably a lower limit on this basis.

I don’t really want to give too much attention to what Nic Lewis says about climate sensitivity, but it is an interesting subject 🙂

How much probability that is located below e.g. 1K for an individual line of evidence doesn’t tell very much about the combined result. For example (I think Annan has used a similar example), assume that my scale tells me that I weigh 80+-0.5 kg. Then I buy some clever new app in the app-store which uses the camera to estimate my weight. It gives 80+-20 kg. The second line of evidence gives a significant probability that my weight is below 70 kg (which is arguably nice 🙂 but the combined evidence is more certain than only the scale estimate, i.e. something like 80+-0.4XX…, so that the second estimate pointed to lower values being possible was really irrelevant. Similarly for EBMs: Taking into account best estimate of bias and structural uncertainty, lets say a likely 1-5K was reasonable at the time of last IPCC reports. Combining with PALEOSENS 2.2-4.8K should give a likely interval starting above 2K (judged, haven’t done any formal calculations..).

Regarding the “physical evidence” and “my basic point was that the physical evidence appears to suggest that the ECS is more likely above 2oC, than below 2oC.”: This is very conservatively put considering that GCM:s as well as the feedback analysis that you link point strongly to >2K. Combining all lines of evidence it is more of like a 90% chance for >2K.

Finally, I think the term “physical” is very problematic. All methods are based on observations interpreted via statistics and physical models in an intricate way. I don’t think it is really meaningful to use the term as distinction in this context.

John L.

Possibly, and I do acknowledge a physics bias 🙂 . I was simply trying to stress that when studying a physical system, you ultimately want to gain understanding of the physical processes that determine the evolution of that system. So, statistical analyses are an important part of constraining our understanding. However, if they produce a result that seems inconsistent with our physical understanding of that system (by which I mean an understanding gained by applying the laws of physics) then you will want to try and understand that inconsistency.

John L.,

Your comment has made me consider something else, that may be related. These PDFs do not really represent the range of possible outcomes, they really represent our ignorance. As we understand it, there really is only one ECS value. So, it would seem to me at least, that this scenario is not one that is comparable to a situation where all the values were possible, and the PDF represented the actual probability of lying within some range. There is only one ECS, we just don’t know what it is. Of course, a statistician may argue that – given our current knowledge – the two situation are essentially equivalent.

There is then an extension to this, which is maybe what you were getting at. If we have a situation like this where we have only one single possible outcome, and a PDF that represents our ignorance of what that outcome is, the instant we can constrain that value in some way, that PDF should change. Therefore there is an argument that could be made that given that multiple lines of evidence suggest ECS > 2C immediately invalidates Nic Lewis’s PDF. He, probably, disagrees 🙂 However, his argument would be stronger if he could actually present a physically-motivated argument as to why his PDF is a reasonable representation of our ignorance, rather than relying on his own analysis to justify the outcome of his analysis.

In summary:

NL’s estimates are incompatible with palaeoclimate evidence and therefore wrong.

🙂

That we also know that he is a politicised partisan with extremely dubious affiliations is of course irrelevant to any discussion of NL’s choice of methodology and the results it produces.

@ATTP

Yes, there is really only one ECS, as a kind of axiom of reasoning. Your reasoning on probabilities is perhaps a bit too sophisticated for me 🙂 I view it more pragmatically, and the “objective”/”subjective” distinction is rather useless and mostly tend to make people confused. Probabilities are just your best attempt to model the likelihood of different outcomes based on what you know. To model this you have to abstract a context (corresponding to a stochastic variable) of similar situations where you at least in principle could test the outcome frequencies empirically. In the end you must make a judgement, there is no escape from that in complex real life situations. But “judged” probabilities follow exactly the same laws as “objective” probabilities, just another way of deriving at the number.

A good example of objective/subjective perspective leading to confusion is Lewis’ attempt to use an “objective prior” as something necessarily more correct than “subjective priors”. Another one is not doing calculations when combining evidence just because you are using partly subjectively derived probabilities (like IPCC seems to have (not) done).

Yes, Nic’s calculated PDF is of course not an overall assessed ECS. If he disagrees he makes a number of mistakes: Cherry-picking not including all evidence, not taking into account model bias and structural uncertainty due to model simplification. A correct simplified way to combine evidence is to take the range where all PDF:s intersect (a bit loosely speaking).

Regarding “physics”, the tricky part with sensitivity is clouds, it is difficult to constrain all possible cloud feedback-effects with only bottom-up “physics”. I think this is one reason why it is not really correct to use that terminology.

John L.,

I doubt it 🙂

Okay, yes, I agree. It is probably not possible (at this stage at least) to do something like clouds from a pure, bottom-up, physics approach. And, clouds are indeed one of the main uncertainties in the modelling. That’s partly why I tend to say “physically-motivated” to imply something that you may not be able to address from a physics only, perspective, but where your procedure is, at least, motivated by your physical understanding.

Analyses so far by the likes of Otto et. al. (2013) and Nic Lewis, who employ Bayesian techniques as being discussed here, rely on temperature series data that finished up at the end of the so-called hiatus in surface temps. I’m betting that if they re-do their analysis in 2016 – with the numbers for 2014 and 2015 included – that their results will change significantly. And that tells you everything you need to know about the validity of their approach.

Of course, I may have this all wrong/too simplistic. But I think their approach relies too much on surface temps, largely ignoring that vast store of OHC that is even now coming back to bite us all in the collective arse.

metzomagic,

I think that this particular paper and the earlier Lewis (2014) paper do also consider OHC. Nic also claims that the hiatus has little effect, but I think that may not be strictly true, especially for the TCR. I’ve just looked at Lewis (2014). Figure 4 shows the non-informative (Jeffrey’s) prior. Unless I’m misunderstanding this, the prior essentially assumes that an ECS 1C [Edit : I had meant to say “peaks below 1C” here]. That does seem like a rather unphysical prior. It would be interesting to see what would happen to Nic’s estimates if he were to simply try a different prior, that was maybe motivated by what is regarded as physically plausible.

Isn’t he claiming his results will not change?

JCH,

Yes, I think he is. Not quite how that can be true for the TCR. If there is an exact relation between and (the planetary energy imbalance) then it could be true for the ECS. However, I think variability studies indicate that internal variability can influence both and , so it may not be true in reality.

Their result produces a 90% range of 1.7 – 4.9oC and a median of 3.3oCAh, that’s the prior distribution, representing the diagnosed 2xCO2 sensitivities of the model ensemble, for PMIP2 using slab-ocean setup and PMIP3 using Andrews et al. 2012 regression method. The observationally-constrained result isn’t hugely different though:

The posterior distribution is shown in red, the bulk of which has been shifted to lower values with the median reducing to 3.0 C. Its 90% probability range has moved slightly less to 1.5–4.7 C.————-

To be honest, I’m not entirely sure I get this, but I think it’s because the vegetation changes and atmospheric dust are feedbacks (not forcings) and so they influence the temperature, without changing the forcings.The issue would be that the temperature observations must contain this additional influence, whereas the models do not. If the models did include these factors they would presumably produce cooler lgm temperatures, further from the observed values, and hence scaling to observations would suggest a lower sensitivity. However, I’m not sure the model 2xCO2 climate sensitivity values include these factors either, in which case the 2xCO2 warming should be scaled upwards equivalently, which would result in a smaller difference compared to the original figures.

There is only one ECS at any point in time. Of course, when things change, there will be other ECSs. If things really change they may be significantly different from the current one. ECS is a one dimensional construct, the Earth is round. While one dimensional models are useful, they are to be used with care.

If Lewis’ prior assumes ECS of 1 K, then he is pulling a Myles Allen on the other end. http://julesandjames.blogspot.com/2009/09/uniform-prior-dead-at-last.html

Paul S,

Cheers, I misread that. You’re right, I quoted the prior distribution, rather than the result.

This is an interesting point

Eli,

That was exactly my thought. People make a big deal about uniform priors, but seem comfortable with one that peaks below 1K.

I’ve just noticed that Things Break discussed this paper and a 2013 update to the LGM temperature reconstruction.

The most physically unrealistic aspect of the prior seems to be that it suggests the most plausible value for climate sensitivity is zero. Now in the absence of any prior knowledge regarding the effects of atmospheric CO2, that might be a reasonable thing to do, but unfortunately we have a couple of hundred years of observations, experiments and theory that shows that this is not the case. While it is true that “The normal scientific methods involves individual studies reporting estimates based on their own data and analysis” if those (statistical) estimates are based on physically implausible assumptions (in this case the prior) and lack physical arguments to provide support, why should we place any confidence in the analysis, when there are others that are based on less obviously incorrect assumptions and which have better support from our understanding of the physics? I would certainly agree that it is a useful contribution to to the subject, but mostly to provide an indication of lower bound on plausible values of CS, however reducing the upper bound on CS would be a much more welcome contribution.

At the end of the day, a Bayesian analysis is only as good as the prior, if the prior is wrong, then it doesn’t really matter whether it is objective or in some sense “minimally informative”, it is still wrong. If we ignore all we know about the greenhouse effect and try and estimate CS from a single realization of a chaotic process, is it a surprise if it gives an answer that is different from that given by a more physically informed analysis?

dikran,

I’m glad you said that because that was what I thought, but it seemed like such a bizarre prior, that I was worried I was misunderstanding something.

Well, yes, this would seem obvious.

It would be quite interesting to take Nic Lewis’s method and simply modify the prior to see what effect it has. I wonder if he makes his code and data available 🙂

Jeffrey’s priors are a very good idea for situations where there is genuinely no prior knowledge about the problem at hand. It may seem bizarre in a case where there is extensive prior knowledge, but it still serves as a bound on what we can reasonable infer from this one set of observations, but it needs to be made clear that the estimate ignores everything that we already know.

dikran,

That’s an interesting point. Could one argue that given that the result using Jeffrey’s prior is statistically consistent with other estimates, that there is little evidence for bias in the other estimates?

I think the thing I would conclude was that the analysis based on the objective Jeffreys’ prior, even though it is heavily weighted towards very low climate sensitivity, effectively rules out ECS below about 1K. To get an ECS lower than that, you would need an

informativeprior that favoured low sensitivityeven more strongly, which would be somewhat hard to justify. As I understand it, negative overall feedback would be pretty much ruled out? That would seem to me to be a useful finding.I don’t think the new analysis provides any indication of a bias in the existing estimates of climate sensitivity, simply because the prior used is (i) obviously incorrect given our actual state of knowledge regarding CS (ii) heavily weighted towards low climate sensitivity (and hence we might expect the lower limit to be lower) and (iii) the prior appears to give very little weight to higher (but considered plausible by the IPCC) values (so the upper limit of the credible interval is also very much lower). It is hardly surprising then that the estimate is at the low end of the IPCC range. Essentially, I suspect that at the upper end, the credible interval may be dominated by the prior, and the observations are essentially overruled. I wouldn’t say it was positive evidence of a lack of a bias though as I don’t accept the prior as being reasonable (even though it is objective, it is not the case that we have no relevant information, which suggest that very low CS is highly implausible).

If the posterior is heavily dependent on the choice of prior, this suggests to me that this set of observations doesn’t provide a very strong constraint on climate sensitivity, and we should be cautious of over-interpreting the analysis in an overly confident manner, unless we can readily agree on the prior.

dikran

Did you mean

positivefeedback? As in ~1K = no-feedbacks ECS to 2xCO2?BBD I meant it ruled out negative feedback, which would make CS less than the 1K from no-feedbacks (at least that is what I was trying to say). If even an analysis that was biased towards very low ECS considers ECS less than 1K implausible, then it is implausible indeed!

Ah, I see. Thank you dikran.

Well, if you believe:

“Forcing agents such as carbon dioxide can directly affect cloud cover and precipitation, without any change in global mean temperature.”

then yes, conceivably no change in temperature is plausible.

Perhaps not likely, but plausible.

TE,

Except then you’re refering to feedbacks that are essentially not temperature dependant. It’s one thing to be plausible, but another to be something that we would regard as actually possible.

My hunch is if you do his analysis with the PDO instead of the AMO, you get very different results.

I’m not sure the model 2xCO2 climate sensitivity values include these factors either, in which case the 2xCO2 warming should be scaled upwards equivalently, which would result in a smaller difference compared to the original figures.Thinking about it, I guess this could be considered encroachment on ESS territory rather than representing traditional ECS.

Turbulent Eddie,

I think you’re misunderstanding that quote. It’s talking about effects of CO2 on clouds which can be considered forcings rather than feedbacks. It’s not about the ultimate impact on temperature change.

From Sherwood et al. 2013:

For example, increasing the concentration of CO2 in the atmosphere affects longwave radiative fluxes and slightly warms the mid and lower troposphere, even with no surface temperature change (see Fig. 2b). In models this subtle change in stratification and relative humidity reduces middle and low-altitude cloud cover (Fig. 3), further altering the TOA net flux even before any global warming or cloud feedbacks take place (Andrews and Forster 2008; Gregory and Webb 2008; Colman and McAvaney 2011; Kamae and Watanabe 2012a;Wyant et al. 2012). This change in cloud cover is quite different to that which occurs subsequently due to the increase in TPual S,

Right. Conceivably, CO2 changes clouds, clouds increase albedo a few W/m^2, and…

..presto – (quasi) equilibrium with no temperature change.

This doesn’t get nearly as much discussion as it should.

First the Sherwood quote goes on:

“AR5 recognises this for the first time, and designates the total forcing including these effects as the effective radiative forcing. ”

The first time?

People have been going on about 3.7W/m^2 as if it were a given and now the IPCC comes out with, “well, we don’t really know what forcing from CO2 will be because the atmosphere changes”.

I don’t doubt that it does, but this is a big deal.

BTW, I have calculated the various complexities of forcing and heating rate for a number of scenarios here.

Much depends on who filled the bucket with what, and how long it was alllowed to ferment

TE,

Okay, but where did it say that this actual scenario was plausible?

Turbs

And …presto, no deglaciation under orbital forcing because no CO2 feedback boosting GAT.

It’s not just implausible, it is incompatible with known palaeoclimate behaviour which requires that CO2 is an efficacious temperature forcing.

“The most physically unrealistic aspect of the prior seems to be that it suggests the most plausible value for climate sensitivity is zero. Now in the absence of any prior knowledge regarding the effects of atmospheric CO2, that might be a reasonable thing to do, but unfortunately we have a couple of hundred years of observations, experiments and theory that shows that this is not the case.”

Its seems fundamentally wrong to use a prior that is physically impossible. I think for maybe 6 months or so I tried to explain to some sleptics ( haha Ill leave that spelling errors) that we could absolutely rule out a sensitivity of 0. but of course they slept through my arguments.

I was debating as to whether or not to make this observation, but have decided in favour of doing so 🙂 It does seem a little ironic that in an era when we’re encouraged not to refer to “climate science denial/climate science deniers”, the preferred method of many “skeptics” is one in which the prior assumption is that the most plausible value for climate sensitivity is zero.

So when other methods produce conflicting estimates of climate sensitivities that Nic Lewis can find ways to cast doubt on that is very relevant to him. But if other studies produce conflicting estimates that he cannot find ways to cast doubt on that is no longer relevant…..

ATTP indeed, it is a bit like using the lack of a statistically significant trend in GMSTs since some particular date as strong evidence that global warming had stopped. The self-skepticim of the null hypothesis significance test is lost if you are arguing

forH0 rather than it being the hypothesis you want to show is inconsistent with the observations in order to be able to reasonably promulgate your research hypothesis.Paul S June 14, 2015 at 12:08 am says: Sherwood 2013 says “In models this subtle change in stratification and relative humidity reduces middle and low-altitude cloud cover”

TE says “Conceivably, CO2 changes clouds, clouds increase albedo a few W/m^2, and…

..presto – (quasi) equilibrium with no temperature change.”

But I don’t see how reduced cloud produces increased albedo, so I go to Sherwood 2013 from Paul S reference, and they say

“Because cloudiness is generally reduced in both cases, these responses produce increased effective radiative forcing and positive cloud feedback, respectively, although the details of the cloud changes vary among models and can be seen here to differ significantly between the adjustment and feedback responses”

Which in my understanding will NOT lead to “no temperature change”

Brian,

Thanks, I was searching for something like that In Sherwood 2013 but obviously didn’t try as hard as you did.

Dikran,

That’s a good way of putting it. As I said above, it would be interesting if it were possible to get hold of Nic Lewis’s code and try different priors.

Dikran,

Actually, what might be interesting is to replace Nic Lewis’s prior, with a prior based on the IPCC’s estimate. That would, presumably, tell you someone about whether or not you can rule out the IPCC’s estimate, using the data/observations used by Nic Lewis.

I suspect the argument against that was that the IPCC estimate was informed in part by the observations used in the Bayesian analysis, so the posterior would double count the information from the observations. Perhaps a reasonable alternative would be to use a prior based on paleoclimate information and see how much it is altered by the instrumental data?

For a more objective approach, it might be argued that a (say) log-normal shaped prior might be reasonable (ruling out both zero and infinite ECS), but where the (hyper-)parameters of the prior are not known a-priori (i.e. they are nuisance parameters to be integrated out). A minimally informative hyper-prior is then put on the hyper-parameters (corresponding to a state of ignorance about the prior, except for its shape) and the Bayesian inference conducted over both the parameters of the model and the hyper-parameters of the prior. A bit more complicated, but I suspect do-able using something like BUGS.

A physical mechanism for low ECS?

Saturation.

http://www.realclimate.org/index.php/archives/2007/06/a-saturated-gassy-argument/

“So, if a skeptical friend hits you with the “saturation argument” against global warming, here’s all you need to say: (a) You’d still get an increase in greenhouse warming even if the atmosphere were saturated, because it’s the absorption in the thin upper atmosphere (which is unsaturated) that counts (b) It’s not even true that the atmosphere is actually saturated with respect to absorption by CO2, (c) Water vapor doesn’t overwhelm the effects of CO2 because there’s little water vapor in the high, cold regions from which infrared escapes, and at the low pressures there water vapor absorption is like a leaky sieve, which would let a lot more radiation through were it not for CO2, and (d) These issues were satisfactorily addressed by physicists 50 years ago, and the necessary physics is included in all climate models.”

As none of the commentators who have commented on prior distributions seem to have much understanding of objective Bayesian methods, may I suggest that they just look at the results I obtain using frequentist profile likelihood methods, which provide confidence intervals without involving any prior? As they are (surprise, surprise), exactly the same as I obtain by objective Bayesian methods using a noninformative prior whose shape several commentators (wrongly) favours low ECS values, I suggest denizens of this blog should also be attacking frequentist profile likelihood methods as similarly biased towards low ECS values.

niclewis: I trust physics before statistics.

“There are three kinds of lies: lies, damned lies, and statistics.”

Nic,

May I suggest that you’re irritatingly condescending and rather annoyingly arrogant, especially given that at least one of those commenting happens to be a statistician who uses Bayesian methods. Given that your modus operandi appears to be to insult anyone who happens to say something with which you might disagree, I’m amazed anyone will interact with you scientifically. Probably proof that most scientists are much more open than your mates at Bishop-Hill and at the GWPF would admit. I note that you still haven’t acknowledged your error with respect to Marotzke & Forster or apologised about the unprofessional tone of your Climate Audit post. I’m not, though, holding my breath.

Where? [Edit : Okay, found it. You do realise this isn’t some kind of Bayesian versus Frequentist issue, though, don’t you? This is about finding a physically plausible explanation for a result suggesting that the ECS is more likely below 2C than above 2C.]

Whining about attacking, when – in fact – all that has happened here is that people have been discussing your work. Any chance you could actually try reading what people write and engaging in an actual discussion, rather than just doing condescending and annoying drive-bys? Also rather ironic given that the norm in your case is to actually attack other people’s work.

In fact, I’ll ask you to explain this

Your prior is Figure 4 in this. How is this not a prior that suggests that the most plausible value for the ECS is 0?

ATTP

“Your prior is Figure 4 in this. How is this not a prior that suggests that the most plausible value for the ECS is 0?”

You rather prove my point about lack of understanding of objective Bayesian methods. As stated in more than one of my papers, noninformative priors have NO probabilistic interpretation. See, e.g., Bernardo and Smith (1994): Bayesian Theory.

“I note that you still haven’t acknowledged your error with respect to Marotzke & Forster”

You refer to what is in your opinion an error; I disagree.

Nic,

And you rather prove my point about being annoyingly condescending and irritatingly arrogant. Seriously, I’m happy to have a serious discussion about this, but if you’re going to behave as you are, I’m going to keep pointing it out.

It is, however, still the prior. It still has a peak at 0. Surely that is relevant, whether or not there is – formally – no probabilistic interpretation? Maybe you could actually answer this question.

No, I’m referring to the fact that , and are either external, or model constants. Therefore if their estimates are reasonable, then how they were estimated is completely irrelevant and the circularity is a non-issue. This is not even a complicated concept.

Nic,

Oh, by the way, you still haven’t explained why you highlighted Lucia’s post in an earlier comment about Marotzke & Forster. I assume it’s because you thought it was somehow interesting. I thought it was nonsense (which may – in some sense – be interesting). Maybe you could explain why you apparently didn’t.

Seems that Pekka got on somebody’s nerves:

http://climateaudit.org/2015/06/02/implications-of-recent-multimodel-attribution-studies-for-climate-sensitivity/#comment-760408

The Auditor usually deletes usages of the F word.

Nic lewis wrote:

‘You rather prove my point about lack of understanding of objective Bayesian methods. As stated in more than one of my papers, noninformative priors have NO probabilistic interpretation. See, e.g., Bernardo and Smith (1994): Bayesian Theory.’

I certainly cannot claim any expert understanding of objective Bayesian methods. I’m trying to begin to learn.

I don’t have Barnado and Smith to hand, but J M Barnardo seems to be regarded as the leading expert.

In, e.g. http://www.uv.es/bernardo/BayesStat.pdf , he makes the following point about interpretation of priors:

‘Reference prior functions are often simply called reference priors, even though they are usually not probability distributions. They should not be considered as expressions of belief, but technical devices to obtain (proper) posterior distributions which are a limiting form of the posteriors which could have been obtained from possible prior beliefs which were relatively uninformative with respect to the quantity of interest when compared with the information which data could provide.’ (p28)

Is this the point that Nic is alluding to? If so, my reading is that this is just a technicality. It is very like saying do not treat a Dirac delta function as a function. Indeed, you do have to be careful mathematically. But physically you won’t go wrong if you think of a very sharp spike. Similarly here, Barnardo seems to be saying we can think of distributions with less and less information that look more and more like the reference prior, but only get the minimum with something that isn’t technically a distribution. Am I right here? If so I think it’s fair to informally interpret the prior as putting weight at zero.

Second, perhaps of more interest is Barnado’s point that:

‘In the formulation described below, far from ignoring prior knowledge, the reference posterior exploits certain well-defined features of a possible prior, namely those describing a situation were relevant knowledge about the quantity of interest (beyond that universally accepted) may be held to be negligible compared to the information about that quantity which repeated experimentation (from a particular data generating mechanism) might possibly provide.'(p27)

In other words, his reference priors are useful in situations exactly where prior information is expected to be negligible relative to information in the data being analyzed. Otherwise, Barnardo seems to be saying, these reference priors are not appropriate. This actually seems sensible. Do I have this right?

In application to climate, I would think that data on observed warming over the last 150 years is certainly a leading source of information. But I don’t think it renders other sources of information (e.g. expectation of nonzero sensitivity) negligible by comparison.

As I say, I’m not an expert, so interested to hear what others think.

JK,

I’m not an expert either, as I think I’ve acknowledged. Certainly what you say seems reasonable to me and is similar to what I was finding when doing some reading about this last night. It does appear as though Nic’s claim that an Objective prior has no probabilistic interpretation is something of a pedantic technicality.

Nic Lewis (or anyone else who understands this properly),

I’m happily willing to confess to ignorance of the prior arts.

It would be most interesting to follow up this brief comment with a reference to the PDFs you obtained using

(a) Frequentist

(b) “Objective” Bayesian

to be crystal clear which of your results you are referring to, and whether the input dataset to these was identical?

You comment “exactly the same” – do you mean that in the mathematically identical sense? Or do you really mean a “very similar” result – I appreciate it makes no practical difference either way, but it would help understanding of the method.

Finally, my (very weak) understanding of Bayesian stats would be that a uniform prior should give the same answer as a frequentist analysis. I take it that’s entirely mistaken?

vtg,

It would be good to get some clarification from Nic. To be clear, if Nic wants to actually engage in a friendly discussion, I’ll stop calling him condescending and arrogant. Of course, that would also require that he stop being condescending and arrogant 🙂

What he’s referring to – I think – are the results in Table 2 of this. However, I don’t really understand how these are determined.

Looking for something on likelihood (compared to probability) in Bayesian statistics I came across this site, which looks as well written as anything I’ve come across on Bayesiany stuff. But before I invest too much time in it, is there anyone who

doesknow what they are talking about who could give an informed opinion on its quality and depth? (My suspicion is that the fact it looks intelligible probably means it is too superficial 🙂 )Thanks

niclewis said on June 15, 2015 at 9:23 pm, in reply to ATTP on June 15, 2015 at 8:13 pm,

“”I note that you still haven’t acknowledged your error with respect to Marotzke & Forster”

You refer to what is in your opinion an error; I disagree.”

…and Then There’s Physics said on June 15, 2015 at 9:29 pm, in reply to niclewis above,

“No, I’m referring to the fact that F, alpha, and kappa are either external, or model constants. Therefore if their estimates are reasonable, then how they were estimated is completely irrelevant and the circularity is a non-issue. This is not even a complicated concept.”

Nic’s claim seems to be that it doesn’t matter how they were estimated – there’s a circularity problem for him no matter what because of some underlying algebraic construction he still can’t see is perfectly valid by group theory. That is, it still seems that he claims that Marotzke & Forster (2015) (M&F) made a purely mathematical mistake in the form of a circular unary mapping, a regressing of a variable on itself, and thus how parameters are estimated does not make the so-called circularity problem go away for him. Perhaps he might now mean that even if the inner mapping in question (see the below) is granted to be binary, then there is still a circularity mistake. But I proved both wrong in a prior comment below: The inner mapping in question is binary, not unary, and so there’s no regression of a variable on itself, and this binary inner mapping and the resulting composition of mappings and composite mapping is valid by one of the basic group theorems. And I prove all this again further below in a simplified proof, since some might benefit from seeing a simpler proof.

In my last comment

https://andthentheresphysics.wordpress.com/2015/06/07/nic-lewiss-latest-estimates/#comment-57630

on June 11, 2015 at 9:35 am under the post “Nic Lewis’s latest estimates”, I addressed all this, and I explicitly wrote out the basic group theorem that ultimately falsifies the circularity claim against M&F, this group theorem being the basis of my prior proof. I linked to my comment

https://andthentheresphysics.wordpress.com/2015/01/31/models-dont-over-estimate-warming/#comment-48402

on February 18, 2015 at 10:33 am under the post “Models don’t over-estimate warming?”, in which I presented a mathematical theorem and proof that essentially shows that a basic group theorem implies the validity of the M&F construction in that it implies there is no circularity invalidity, and so this circularity claim against M&F is not only a direct contradiction of a consequent of this group theorem, it is by modus tollens a claim that this group theorem itself is false (even though this theorem is of course true).

For the reader at large, since some may not follow what I said in my theorem and proof in my earlier comment above, here’s an outline of a simplified form of my prior proof in a nutshell, simplified to k-ary functions restricted to k = 1 or k = 2:

First, define all variables to be elements in a commutative group G (recall that the real numbers form a field, which forms a commutative group under addition and whose nonzero elements form a commutative group under multiplication). Now these preliminaries: We can from the binary operation (binary function) over G define a unary function. That is, by the convention of taking multiplication as the standard presentation of this binary operation, we can from the binary function m(w,x) = y = wx, which we can represent as w,x -> y, define a unary function f(x) = y = wx, which we can represent as x – > y. Let’s use a different letter of the alphabet than w to denote that in the unary function f the value of w is fixed while the value of x varies, and so let’s write the form that is familiar to all algebra students f(x) = y = ax. (This unary function we teach to algebra students obtains by this derivation, by the way, in the context of teaching them that multiplication on the real numbers and its various familiar subsets is a binary operation.) Finally, recall that we can speak of the domain of the output variable as well as of the input variable(s) of a unary or binary function.

Theorem: For any unary mapping x -> y = f(x), we can construct a binary mapping y,z -> y = h(y,z).

Proof: By that fundamental theorem on groups that I explicitly wrote out in my more recent comment above, for each x,y in the mapping x -> y = f(x) there exists a unique z such that yz = x. Going through all such x,y in this manner yields a set of ordered pairs {({y,z},x)} that meets the definition of a binary function g(y,z) = x = yz, which we can represent as the mapping y,z -> x. We now have a composition of functions such that the outer function is the unary mapping x -> y and the inner function is the binary mapping y,z -> x, and from this composition of mappings y,z -> x -> y we have the composite mapping y,z -> y = h(y,z). (Note: This last expression does *not* say yz = y. For a binary function, it’s simply a mapping of the input set of variables to the output variable.) The composition of functions and composite function in question is *always* valid given that the inner function is obtained from application of the binary operation over the group, as was done above. (For a group under addition, simply make the appropriate changes to the notation.)

QED

Now I’m going to go even further than I did in my proof in my prior comment.

Suppose we want to go back and define unary function f to be a partial function (although the definitions given already allow for that) in which the domains of variables x and y are proper subsets of G. This means that, generally speaking from a purely mathematical standpoint (recall that according to Nic this is about pure math, nothing to do with physics), we can have a number of possibilities depending on the formulas that contain the variables x,y,z. One of these possibilities is to have the domain of y be a subset of the domain of a new variable in the position of y in y,z -> x, this new variable being something like y’. Running through the above proof with this substitution this would give us y’,z -> x -> y and y’,z -> y, of which y,z -> x -> y and y,z -> y would be a subset. In this way, y’ and y could generally denote the same underlying object with the only difference being that the denotation of this object with y is under a restricted domain, at least compared to the denotation of this object with y’.

This basic group theorem via compositions of functions allows us the freedom to create all kinds of perfectly valid constructions no matter how complex or seemingly invalid via seeming circularity problems, even when the same mathematical object (or physical phenomenon) is represented in different parts of the construction with the same variable or with different variables even when their only difference is that the domain of one is a subset of the domain of another.

The M&F construction is one of these perfectly valid constructions. To see this explicitly, just make the appropriate variable substitutions using the M&F variables into the above to see this, or, since I explicitly made these substitutions in my earliest comment above in February, see these substitutions there.

Nic writes

That’s true by construction and proves absolutely nothing.

The way Nic formulates the problem means that he chooses the prior to make that true, but that’s exactly where the frequentist method (as Nic uses the concept) fails to reveal the subjective choices that are an essential component of the Bayesian method.

The frequentist method cannot really be used properly for this kind of problems. All the data is used to get the estimate of the climate sensitivity. Thus we cannot obtain several independent determinations of that, and we cannot determine the ranges where 90% or 95% of the obtained values fall.

Pekka,

Thanks, I did wonder if that was the case but – as Nic has already pointed out – I don’t really understand Objective Bayesian methods 🙂

If the climate sensitivity were the only unknown parameter we could describe, what each method produces as the likelihood or conditional probability that can be derived from the empirical data. In that case the choice of prior would form a fully transparent and independent input to the derivation of the posterior PDF. (The prior and the likelihoods would be simply multiplied to form the posterior PDF).

What makes the situation more complex is that there are other parameters (typically ocean diffusivity, and something that tells on aerosols). To get the PDF of climate sensitivity, we must integrate over the other parameter, and to do that we need priors for them. The multidimensional prior is not any more a simple multiplicative factor. Nic’s approach takes that into account, but the problem is that we cannot tell, whether his priors provide proper weights for various parts of the multidimensional distribution or cause severe bias.

Nic has proposed that the priors should be of Jeffreys’ type in the variables that describe empirical data rather than model parameters like climate sensitivity and ocean diffusivity. Such choices make the priors dependent on the way the empirical data is handled or possibly on the technical limitations of the empirical work. That’s, however, not a valid criterion when we are looking at the determination of model parameters of a physical system. Nic’s choice leads to severe incoherencies although he has claimed the opposite as an argument in support of his approach.

Bayesian credible intervals and frequentist confidence intervals are answers to two different questions. Often modelling choices can be made that mean the two coincide, but even then they don’t mean the same thing.

A Bayesian credible interval is an answer to the question “give me an interval which contains the true value with probability (say) 0.95”, which is what most scientists would actually want.

Frequentists define probabilities as long run frequencies, which mean they can’t attach a non-trivial probability to something that doesn’t have a long run frequency (for instance the true value of the statistic either is within an interval or it isn’t, so its long run frequency is either one or zero, but we don’t know which), so they use the concept of a confidence interval. This is the answer to the question “give me an interval, such that (say) 95% of intervals computed using the same method would include the true value if you repeated the analysis a large number of times with different samples of data from the same underlying distribution”. This is not really what the scientist actually wants to know, it is a statement about some (possibly fictitious) population of experiments that the scientists could have performed, rather than a statement of what we can conclude from the results of the experiment that actually was performed. In this case, we have only on set of observations from the Earth we actually live on, so the meaning of this “population of experiments” is a little obscure.

The key point is that there is not necessarily a 95% probability that the 95% confidence interval you compute for a particular experiment will actually contain the true value. Indeed it is possible to produce a properly constructed 95% confidence interval in which you can be 100% sure (given the data) that the true value lies nowhere within the interval (see here). That might seem like a contradiction, but it isn’t because it is treating a confidence interval as if it was a credible interval, which is a classic statistical error. Comparing confidence intervals and credible intervals is something that needs to be done with some care, and with the necessary caveats.

> as Nic has already pointed out – I don’t really understand Objective Bayesian methods

There was no need to parse how you interpreted “noninformative priors” to discover that, AT. Reading your first article might have sufficed:

https://andthentheresphysics.wordpress.com/2015/06/07/nic-lewiss-latest-estimates/

This seems to indicate that Nic has not really read your post.

***

Since we’re into nitpicking territory, here’s James Berger, from the Bernando & Berger duo:

http://projecteuclid.org/download/pdf_1/euclid.ba/1340371035

If Nic could cease and desist his usage of “objective,” that would be nice.

I have some problems understanding what the point is using a prior for climate sensitivity at all. OK, maybe I should just study Lewis’ and others work more carefully before commenting 🙂 But lacking the time I give my thoughts on the issue anyway…:

You have a physical model, an EBM, + some empirically-based estimations of historical forcings and temperature (and possibly some other parameters), where climate sensitivity is an emergent property that can be derived from the empirical variables. Because of the simplicity of the physical model you can include the uncertainty in all empirical variables in the analysis and integrate over their PDF:s. From that calculation you get a PDF of the climate sensitivity which should tell you something interesting. Is this what Nic Lewis call “frequentist profile likelihood methods” above? In any case, it would be a quite straightforward way to see what EBM:s + the empirical data really tells you. Next step is to evaluate the physical model, e.g. is the lambda really constant and what does that mean? Do you need corrections for forcing efficacy? Etc. If you involve any ECS/TCS priors from feedback analysis or paleo before that analysis, I think you are just mixing things up and it gets unnecessarily messy like the discussion above shows.

I add a few comments on Jeffreys’ priors and reference priors, which are closely related.

The idea is that prior is chosen in such a way that the data is expected to have maximal influence on the posterior. To reach that goal largest weight is given to those regions, where the empirical method used has largest separating power and zero weight to regions where the method has no separating power. The amount of separating power is estimated based on some model, but that model may be a model of the empirical method rather than of the physical system itself.

A good example of the failure of that approach is radiocarbon dating. Due to variations in historical isotope ratios, there are periods over which the method cannot separate different ages. Jeffreys’ prior based on the method of radiocarbon dating tells that those real ages are virtually impossible, which is an obviously nonsensical result. The probability of such an age of a sample cannot be controlled by the weakness of the method of analysis to determine the real age within a particular range.

In the case of the estimation of ECS such special problems are not expected, but the overall shape of the distribution based on the separating power of the method may very well be biased in any direction. From the paper Lewis (2013) (Fig. 4) we know that the resulting prior used in that paper has a strong cutoff both for high ECS and for high effective ocean diffusivity. Such cutoffs are plausible also for other reasons, but it’s impossible to tell, whether the strength of those cutoffs is appropriate.

What makes the choice of prior so controversial in this case is the limited separating power of empirical data and the related very large relative uncertainty in the parameter values. The prior may have more influence on the outcome over wide range of values than the data. Under such circumstances the preconditions for using (rather) safely reference priors are not satisfied.

It’s often said that, when the reference prior is used the data is allowed to speak, but as I describe above that’s achieved by excluding alternatives, where the method has no or very little power. The limitations of the empirical method are given a large weight in a way that may be totally inappropriate. The prior is not determined by the data itself, but by the power of the empirical method to use that data.

Pekka,

Thanks, interesting comment. Maybe Nic will come back and address the points you’ve made?

As I think I’ve said before, Bayesian methods coupled with enough independent pieces of data will produce essentially the same estimate for a quantity of interest no matter what your original prior (as long as it’s not zero or ridiculously tiny where the real answer is). The beauty of the Bayesian approach is it drives differing preconceived notions about a system toward a consensus on what reality is. But getting there requires pulling in all the relevant pieces of information. If you leave something out (like the basic underlying physics as a separate, independent, piece of information about the system), then you are a step or two behind on that approach to reality. Over time as we gather more independent observations or other information, estimates should improve. If it’s a system where there’s significant initial disagreement on what the “prior” should look like and we don’t have enough data to narrow things down much, there’s really not much you can do other than up your uncertainty intervals to include the full range that people accept. And uncertainty here is not our friend…

Well, thanks Pekka et al.

I now feel really ignorant, and am more convinced than ever that I don’t understand this properly.

As I don’t understand it, I think I shall continue to defer to the consensus of the experts, that ECS is likely in the range 1.5-4.5, and that Nic’s estimates, whilst plausible are very much at the low end of all the possible methodologies.

I hereby retire to the cheap seats at the back with my popcorn.

Take that, Dunning and Kruger!

vtg,

I find myself reading some of the comments over and over again, to try and get the subtleties. I may understand Bayesian statistics one day 🙂

Following on this fascinating wordology, Radford Neal mentions a conflation that deserves due diligence:

http://www.cs.toronto.edu/~radford/res-bayes-ex.html

An alternative to the philosophically incorrect “objective Bayes” could therefore be

arbitrary Bayes.Another alternative would be

pseudo-Bayes:Op. cit.

Neal is a bit more severe than Berger on pseudo-Bayesians.

In any case, it would be interesting to know which guarantees Nic’s specific prior provides.

Everyone knows what bad Bayesian statistics looks like.

Apparently Microsoft cranked the frequency of help detection up at the last minute for our dear friend Clippy. The result has been decades of jokes;

https://en.wikipedia.org/wiki/Office_Assistant

Another nit:

That’s a tad weaker than Nic’s caps locked “NO”.

Does it mean he does not really understand Arbitrary Bayesian methods?

A comment by Dawid on that Socratic dialog:

http://www.sciencedirect.com/science/article/pii/S0378375897900690

Seems that there’s a problem with the likelihood principle.

When you want to know what GCM:s tells you about climate sensitivity, you just run them and get a number out. It is completely irrelevant what your opinion/prior of ECS is and you try to not involve external factors in experiments and analysis. Later you may start investigating whether the results are plausible etc.

Why should EBM:s be different? Remember, as ATTP also wisely hinted in the headline, they are physical. The Bayesian approach is technically interesting but it doesn’t add anything valuable (only confusion) until after you have run the analysis of the model + data and want to understand if the model is biased and what structural uncertainty there is. At this point you may start to compare with other evidence to get a hint for what factors and possible errors in the model you should look for. Or later when you want to combine all the evidence, this is when Bayesian statistics is useful.

John L.,

That’s a good point. There’s something that I don’t think is fully appreciated. The ECS, for example, is actually a model metric. As you say, you can run a model increasing CO

_{2}until it has doubled and then let it run to equilibrium. That would give you the ECS. However, there are – as I understand it – models with quite high ECS values, that fit the instrumental temperature record quite well. That would seem to imply that the Effective Climate Sensitivity one would estimate from such a model (using only the instrumental temperature record period) would be different to its Equilibrium Climate Sensitivity. I don’t think anyone has actually checked how these two values differ across the different models.With regards to “physically plausible” models, take a look see at the state of ENSO and El Nino model. The most I was able to find was that about the Zebiac-Cane model was that it “produced plausible simulations of ENSO”. Yet there are no ZC models fitted to time-series data that demonstrates plausibility.

Pingback: Bayesian estimates of climate sensitivity | …and Then There's Physics

After the hockey stick debacle and Mark Stein’s book of quotes about you how do you assess your credibility in this field?

Martin,

I’m not aware that Mark Steyn has written a book of quotes about me.

Pingback: Prospects for narrowing ECS bounds | …and Then There's Physics