It seems that some climate “skeptics” are revelling in the publication of another hoax paper. In this case a paper called The conceptual penis as a social construct, which has now been withdrawn (an archived version is here). The authors have written about it in this blog post, and say
The publication of our hoax reveals two problems. One relates to the business model of pay-to-publish, open-access journals. The other lies at the heart of academic fields like gender studies.
Well, the first seems a reasonable inference, but suggesting that this somehow reveals a problem at the heart of academic fields like gender studies seems a bit of a stretch. It might indicate that there are some fields where complete nonsense can be made to sound scholarly, but that doesn’t mean that there is necessarily some fundamental problem at the heart of those academic fields (I will admit, though, that if sounding scholarly is valued more than clarity, then it might be worth rethinking what should be valued).
What is more, the paper was actually initially rejected and then published in a journal that they themselves acknowledged may have a slightly suspect business model. It would be highly unlikely that a single paper could ever reveal a problem at the heart of an academic field, but even less likely if it’s a paper that requires a bit of shopping around to get it published.
Anyway, why do some climate “skeptics” seem to like this story? Well, one reason is that the paper mentions climate, but another is that it allows some, like Matt Ridley, to argue that peer review of science is a deeply tainted system. Peer review is not perfect, but that some people – who try hard enough – can get a rubbish paper published in a second-rate journal doesn’t suddenly make it tainted. This isn’t to say that we couldn’t improve peer review, but (like democracy) peer review is probably the worst possible system, apart from all others.
However, what I found interesting in Matt Ridley’s article was a quote from one of the paper’s authors:
“The academy is overrun by left-wing zealots preaching dangerous nonsense,” says Boghossian. “They’ve taught students to turn off their rational minds and become moral crusaders.”
So, someone who has published a fake paper to try and highlight some kind of fundamental bias in the academy, can’t even hide their own bias. As Ketan Joshi says in this article
Shermer, Boghossian and Lindsay inject a strong current of mean-spiritdness into their hoax, far removed from any effort to shine a light on unethical practices in publishing. Issues around rape, identity and sexuality are weird targets for sneering derision, alongside climate change action.
They’ve perceived the shape of political and moral bias in an entire field, based on a single pixel of information.
As far as I can tell, the authors are not two independent observers, but are more two people who have a problem with some of what they see in some areas of academia and so have constructed the evidence that they’re then using to argue that there is a major problem. Hardly an unbiased way to illustrate the existence of a bias.
...and Then There's Physics 
It was just a predatory pay-to-publish vanity journal with no meaningful peer review process. Lots of people have published more blatantly bogus articles in that type of outlet.
James,
Indeed, like this. If you’re going to infer something about a field then you should at least aim to publish in a flagship journal, not in some journal that is highly unlikely to reject what you submit. I’ve also just noticed that it appears to be less than a month between submission and acceptance which would suggest a rather lax reviewing process.
The penis paper wasn’t a spoof. Those who say (Griffin, 1989) that it was (LaDuke, 2002) are tools (Bordo, 1999; André 3K, 2003) of the white (Sjoo et al, 1991) patriarchal (Sheidlower, 1999), plutocratic (Lerner, 1986) hegemony (McClintock, 1995) – of ‘the White Man fucking the Dark Mother’ (Caputi, 2016).
Sure, their second point can be easily critiqued — more or less easily, depending on each readers’ worldviews entering into the question. I honestly don’t have any opinion about how much their paper bears on that question, as I would need to be comprehensively informed about the current state of affairs there and I am decidedly NOT well-informed. So far as I’m concerned I’d just grant you the point and move on.
Their first point, though, ties in with other comments I’ve seen you make. And it’s a valid criticism. Not that it will “fix” anything. So long as there remains a paying marketplace for faux-news and faux-science, or else valuable political leverage to be achieved out of a general ignorance of the differences, there will exist such avenues.
I suppose, in summary, the article and its efforts achieve very little. It’s not going to change anything related to their first point, as will only preach what is already known to a narrow choir that already gets the point, anyway. And the second point would require a thoroughly well-planned and well-conceived question to ask, along with a specific, viable methodology in answering it, plus the added time and money required to pursue it. And this paper didn’t do that, either.
On one hand, it’s just a slap in the face towards Cogent Social Sciences, I suppose. On the other, for those with a voice to an eager and gullible public, it can be bent to any service they desire. But that has always been the case. Even good quality articles published in good quality periodicals get their words easily twisted and made into something not only entirely unintended by their authors but almost the diametric opposite of it. I think you’ve made that point from time to time, as well. So what else is new?
It’s just same o ‘ same o ‘. Oh, well.
aint nothing blockchain cant fix, just tokenize the peer review process.
http://userfeeds.io/
“I’ve also just noticed that it appears to be less than a month between submission and acceptance which would suggest a rather lax reviewing process.”
Have Shermer et al published the peer review comments? If not, they should. That will immediately show whether their paper went through the usual sham peer review at predatory open access journals, or through a real peer review.
It seems to me that a large part of the problem is that the general public don’t seem to have a good idea what the peer review process is actually for. It isn’t intended as means of ensuring that papers that make it into the journal are reliable (if it were many climate skeptic papers would not have been published, e.g. Essenhigh, Humlum, Harde), the reviewers simply don’t have the time to spend checking the work that thoroughly (and often that is impossible anyway, for instance because reproducing the result would take a processor-century of computation). It is better to regard it as a sanity check filter to try and ensure that most of the papers published in the journal are worth reading (at least to some of the audience), while at the same time not rejecting too many ideas that, while not completely supported, may prove to be of some value at a later stage. The filter has been good enough for Murry Salby not to have been able to publish his theory in a journal (I suspect his academic standards are high enough not to stoop to a vanity open access publisher – if so, good for him!). Being operated by humans, the filter is bound to be imperfect, so sometimes there are failures (see above), but that doesn’t mean anything is “broken”. In short, you might think something is broken when it isn’t simply because you misunderstand what it is actually for.
If there is part of the system that is broken, I would suggest that it is that some journals ask the author for suggestions of suitable reviewers, which is an invitation for “pal review”, which I suspect may be a partial explanation for the three failures of the review process mentioned above.
The loss of Beale’s list of predatory journals is not going to help things much.
I thought I would look up the academic credentials of the two authors (Boghossian and Lindsay). As far as I can tell Boghossian has published about 7 papers and been cited about 53 times. James A Lindsay may never have published a paper, although it’s hard to tell as there are quite a few people with the same name, but I can’t find anything that might be his.
So, you have two people who aren’t even close to being leaders in their own fields, thinking that they can illustrate a problem at the heart of another academic field. I realise that the former doesn’t imply that the latter isn’t possible, but it does suggest that they don’t really have the background to even begin making the claims that they’re making.
“Indeed, like this. If you’re going to infer something about a field then you should at least aim to publish in a flagship journal, not in some journal that is highly unlikely to reject what you submit.”
It’s not quite as clear that “Cogent OA” was known to be a predatory journal. Any objections to this journal seem to have come after the fact. Although they do have an interesting policy about manuscripts not suitable for their flagship journals: “we also provide authors with the option of transferring any sound manuscript to a journal in the Cogent Series if it is unsuitable for the original Taylor & Francis/Routledge journals”
Moral of the story is, don’t be a dick
lerpo,
My point was more that if they’ve moved once, would they have moved again had it been rejected from the second journal? That they managed to publish it doesn’t really say anything about the field, especially as this wasn’t really a Gender Studies specific and we have no idea who the referee was, or how many there were. Even good flagship journals can send papers to lazy reviewers, or to reviewers who don’t know the field as well as they should.
Roger,
That’s a rule that could be applied in many circumstances.
“Even good flagship journals can send papers to lazy reviewers, or to reviewers who don’t know the field as well as they should.”
A certain 2009 JEP paper comes to mind…
I liked this post:
(H/T Michael Brown, tweet. comm.)
I actually included that in my post.
That’ll teach me to stop reading after I get your point and only to restart reading when JamesA or RogerJ comments.
I might as well state the critical point of constructionnism:
That the penis is subject to a social construction should go without saying. Showing’s enough:
I hope that social constructionnists’ [3] doesn’t imply castration.
“That’ll teach me to stop reading after I get your point and only to restart reading when JamesA or RogerJ comments.”
As always: Read the first paragraph, then the last.
From that you should be able to hypothesize the argument made in the middle..
but you knew that
But less is more…
The rise of Critical Theory has been attended by the creation of some dozens of small journals to allow PC postmodernists to avoid perishing by repeating each others voluminous cant .
What’s terrifying is that authors could just as easily have gotten it published for free – nobody had to pay page charges to give us Feminist Glaciology , Geontology, or No Failure :Climate Change, Radical Hope, and Queer Crip Feminist Futures
I think I’ve passed posting a dozen examples of the Too Gonzo For Sokal genre so far., eg:
https://vvattsupwiththat.blogspot.com/2017/01/does-this-mean-more-or-fewer-bathroom.html
Peer review from a small clique each supporting the other is not tainted?
At least the guys getting this rubbish published had to pay for their review.
Not be given it gratis.
angech,
Apparently, they didn’t ever pay.
“Peer review from a small clique each supporting the other is not tainted?”
like Energy and Environment, Pattern Recognition in Physics and likely the Open Atmospheric Society Journal (if it ever gets off the ground, but at least they have the opportunity to learn from the mistakes of pRIP)? As I mentioned above, I suspect the reason why a few deeply flawed climate skeptic papers manage to get published is they opt for journals that request the author to suggest candidate reviewers, and they give name from their own “small clique”.
“At least the guys getting this rubbish published had to pay for their review.
Not be given it gratis.”
No, if you publish in a predatory open access journals, you are almost certainly not paying for the review; the reviewers of open access journals are not likely to be paid either.
“the reviewers of open access journals are not likely to be paid either”.
Well……….if it is a real predatory publishers, there’s a good chance the “peer review” is done by the same person who runs the whole show, and so does get paid.
Marco, indeed, which of course explains why I so rarely get asked to review papers for predatory open access journals, compared with the frequency of review requests from mainstream journals. ;o)
Tainted review? … lol.:
When you fish there’s the fishes you barely hooked, and then there’s the fishes that swallowed everything.
The fishing season is indeed upon us and at least one Bloombergen stringer has hooked into a whopper :
https://vvattsupwiththat.blogspot.com/2017/05/scaring-fishes.html
Angech.
A token system that cut out journals would solve everything.
Imagine a system where you had to pay a small amount to cite a paper. ..to the author of the paper.. and a system where reviewers became the publishers and collected payment from readers.
Here’s Poe’s Law:
We could generalize this law. Imagine the perfect Poe, i.e. a piece or a paper that pastiches some genre so well that it’s totally indestinguishable from its sincere target.
May I ask why this perfect Poe shouldn’t be accepted as a valid piece of work in that field?
I think it should, otherwise the authors’ intentions become decisive in accepting or rejecting their work. Think about it: is Kepler’s law true only because Kepler really meant what he was writing
PS: Here’s my own imperfect Poe.
Or just a system that cut out the commercial publisher entirely and was free for both authors and readers. The minimal web-hosting and administrative costs can be carried by the universities that gain prestige by hosting them, libraries who want a paper copy for archival purposes can do so via print on demand services. In an field where the bulk of the work is done by academics on a pro-bono basis, there is no clear need for or benefit in any commercial involvement AFAICS.
Paul Romer strikes again:
https://www.bloomberg.com/news/articles/2017-05-25/war-over-words-erupts-as-world-bank-star-economist-is-sidelined
The same should apply to critical theory, perhaps to the dismay of our Russell.
Since more than one journal features the same dog on its editorial board, why not train it to bark at bad papers and add it to the Reviewer’s List?
https://vvattsupwiththat.blogspot.com/2017/05/so-why-didnt-she-sign-oregon-petition.html
Russell, why mention Holt and Nye (previous item to your link)? The rest of this is amusing: https://www.youtube.com/watch?v=QyIDuBBXyC8
Time for scientists to get regarded as heroes. What a hope, but I think we’re going about this all wrong, just getting wonkier and wonkier and piling up evidence only makes things worse. Unfortunately, Throbgoblins appears to have disappeared along with wonderful Marc Roberts, but here is “Trick”
Susan, my first match choice was former Rep. Rush Holt v. Rush Limbaugh, but that wouldn’t fit the headline and Holt would get Sumo’d
Then I recalled this apparent out-take from Wrestlemania:
https://vvattsupwiththat.blogspot.com/2017/04/henry-adams-was-not-available-for.html
Russell, you outclass me in word play and sophisticated references, but it is indeed a pleasure … One could wish that Bill Nye didn’t overplay himself, given his mien.
I’m glad Bill didn’t get his hands on a folding chair.
The problem is not so much that bad papers slip through peer review, but rather that bad papers, or bad behaviour, by influential academics is not called out.
Here’s an example: http://sticknern.blogspot.co.uk/2017/05/credit-where-its-not-due.html
@ Susan, Marc Roberts seems to have given up cartooning on personal websites, but draws the Only Earth monthly cartoon feature in New Internationalist magazine.
https://digital.newint.com.au/categories/172
Magma, wunderbar, thank you! I met him and know a bit about his troubles, and am glad he is still around.
And then there’s Chris de Freitas. A blast from the past.
“Or just a system that cut out the commercial publisher entirely and was free for both authors and readers. The minimal web-hosting and administrative costs can be carried by the universities that gain prestige by hosting them, libraries who want a paper copy for archival purposes can do so via print on demand services. In an field where the bulk of the work is done by academics on a pro-bono basis, there is no clear need for or benefit in any commercial involvement AFAICS.”
The Key is incentivizing the behaviors that help to create trust. For Starters I would establish a system of Anonymized IDs for reviewers.like a DOI. So when I read a paper I see that Reviewer
X, Y and Z, signed off on the paper. I can then look at the track record X,Y and Z have. other papers they have accepted. You could also have folks sign rejections as well.
Imagine if you would a reputation system for reviewers. Anonymized and still verifiable.
You could do the same thing for readers and authors. All anonymous. All traceable. And really interesting data when you look at “approval/disapproval” graphs.
Once such a system was established, you could choose to monetize the activity of reviwers or not. and you could do the same thing for readers.
Its a weird metaphor but Im also thinking of ways you turn citation into a “transaction”
1. when I build on your work I am Giving you value
2. When I criticize your work I am taking away your value.
Much harder to implement, but one thing a reviwer would do is check/audit your “transactions”
This is basically science publishing as a block chain.
Still a metaphor
Poe?
Too easy. (Who would spend time writing nonsense like that if they weren’t sincere?)
> Who would spend time writing nonsense like that if they weren’t sincere?
Good question. Ask our dynamic of Freedom Fighters, Peter Boghossian, Ed. D, and James Lindsay, Ph. D.
My question was rather: why should we care that most of Newton’s works have been written from the perspective of a mathematical mystic?
When I feed your pastiche (“white fallus,” srsly?) into the G search, here’s the first hit I get:
How does that work for you, Vinny?
Richard,
The problem, as I see it, is that when people call out bad behaviour, by influential academics, the influential academics whine.
> The problem is not so much that bad papers slip through peer review, but rather that bad papers, or bad behaviour, by influential academics is not called out.
Gremlins.
Done.
Next.
I like this approach:
–snip–
Novel empirical insights by their very nature tend to be unanticipated, and in some cases
at odds with the current state of knowledge on the topic. The mechanics of statistical inference
suggest that such initial findings, even when robust and statistically significant within
the study, should not appreciably move priors about the phenomenon under investigation.
Yet, a few well-conceived independent replications dramatically improve the reliability of
novel findings. Nevertheless, the incentives to replicate are seldom in place in the sciences,
especially within the social sciences. We propose a simple incentive-compatible mechanism
to promote replications, and use experimental economics to highlight our approach. We
begin by reporting results from an experiment in which we investigate how cooperation
in allocation games is affected by the presence of Knightian uncertainty (ambiguity), a
pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find
that adding uncertainty enhances cooperation. This surprising result serves as a test case
for our mechanism: instead of sending this paper to a peer-reviewed journal, we make
it available online as a working paper, but we commit never to submit it to a journal
for publication. We instead offered co-authorship for a second, yet to be written, paper
to other scholars willing to independently replicate our study. That second paper will
reference this working paper, will include all replications, and will be submitted to a peerreviewed
journal for publication. Our mechanism allows mutually-beneficial gains from
trade between the original investigators and other scholars, alleviates the publication bias
problem that often surrounds novel experimental results, and accelerates the advancement
of economic science by leveraging the mechanics of statistical inference.
–snip–
Things get into journals that should not. Most of the time, such papers are simply ignored. Sometimes an erratum is in place, sometimes a retraction.
We should worry about those papers that are wrong and influential, but not corrected.
We should worry about those academics that do wrong but are not corrected.
In other words, Ridley’s extrapolation from two papers misses the mark. The number of papers published per year is now measured in the millions. Of course there are papers that go wrong.
Richard,
Do you have an example of a wrong, but influential, paper that has not been corrected (and, to be clear, I don’t mean a paper that simply think is wrong and influential).
Okay, but how is this meant to work? If someone has supposedly done wrong but hasn’t been formally disciplined, how is the community meant to deal with this? Maybe, more specifically, how does the community do something without ending up being accused of a witchhunt, or a smear campaign?
=={ In other words, Ridley’s extrapolation from two papers misses the mark. The number of papers published per year is now measured in the millions. Of course there are papers that go wrong. }==
A sensible comment. Whoda thunk it?
Seems to me the most egregious and misguided gender studies paradigms are not widely influential – but are given much more attention than they deserve simply because, for mainstream media it is a popular button pressing/reaction invoking entertainment to focus derision on them. It’s a format I’ve seen applied to media panel discussions of climate related science; find the most nonsensical utterances (someone somewhere can be guaranteed to make some, whether they are truly nonsense or just counter intuitive and appear to be), and make mock of it. And, whether deliberately or carelessly, taint the serious, credible advice with the implied associations.
This attempt by Shermer et al to mine the predictable reader reactions to make something more than mocking entertainment and make a serious point about peer review required – as so many other commenters note – it be successfully passed through real peer review of the serious and credible science publications, not a vanity publisher. I find the Mainstream media playing that game a bit tawdry – and the credibility of the conspirators in this are tainted by that unwelcome association.
Speaking of mining predictable reader reactions, a blast from the past:
http://grist.org/climate-change/2011-04-26-why-ive-avoided-commenting-on-nisbets-climate-shift-report/
Freedom Fighters do that kind of thing since the dawn of time, i.e. The Road to Serfdom.
Cue to Richie’s concerns, I mean worries.
“bad behaviour, by influential academics”
“We should worry about those papers that are wrong and influential, but not corrected.”
Now there is a subject area where I think that a certain person posting here definitely has a lot of experience and expertise.
@wotts
A recent example for a paper is Burke, Hsiang and Miguel (Nature, 2015).
A recent example for a scholar is Stern. See link above.
Richard,
Again, I meant examples that are definitively true, not examples of things *you* think are wrong and haven’t been corrected.
A recent example is Tol (2015) which contains several errors that Richard didn’t want to discuss.
SM most academics are not primarily driven by financial gain; monetizing rewards is unlikely to encourage the correct behaviour (more likely [IMHO] to be exploited/gamed by those who are driven by financial gain, rather than inquisitiveness). A system where researchers were rated according to the quality of their reviews (without it being made known which papers they had reviewed) might be more effective. However I don’t think there is enough of a problem with the review system at the moment to motivate a large scale change.
Having said which, a journal once gave me a book voucher for performing a review, which I liked.
Dikran,
To me, the problem is that we simply publish too much. The same people who are driven to publish more are also those who undertake the reviews. If we published a bit less, we might have papers that had more value, and we might have more time to undertake reviews.
@wotts
Reasonable people can indeed disagree whether it is acceptable to puff up the final report to a funding agency by a factor four.
As to Burke, it is true (in the mathematical sense of the word) that an I(1) process cannot cause an I(0) process.
ATTP, yes, that is exactly the problem. I suspect I’ve said this before, but one of the good things about RAE/REF in the UK is that your contribution is judged by your best four papers over the assessment period, which rewards quality rather than quantity of output. I suspect it also discourages salami-slicing if your best four papers are minor variations on the same theme, it would probably be rated lower than four papers on for distinct topics.
Richard,
I have no idea as to whether or not what you say about Stern’s report is even correct. I’m not about to trust the analysis of a site that references Principia Scientific.
Then it should be easy to publish some kind of response that corrects what you suggest is an obvious error.
Susan Anderson says: Throbgoblins appears to have disappeared along with wonderful Marc Roberts, but here is “Trick””.
While I appreciate the cartoon, very funny, there is a deeper problem here.
No, peer-review isn’t tainted! was the title of this blog.
Some people believe that the trick referred to by a group of peers was not bending the legs.
Now it might be in the nature of men to be a bit tricky at times but it did leave a bad impression.
Peer review was used in subsequent papers used to bolster the original claims.
“The Key is incentivizing the behaviors that help to create trust. For Starters I would establish a system of Anonymized IDs for reviewers.like a DOI. So when I read a paper I see that Reviewer
X, Y and Z, signed off on the paper.”
I, for one, would love to know who the two anonymous reviewers were on that follow up paper.
It is not true that zero is a positive number, which is important in a paper that claims:
IIRC the one study that had a genuinely positive impact was one of Richard’s earlier papers! ;o)
“Then it should be easy to publish some kind of response that corrects what you suggest is an obvious error.”
Our advice was the opposite.
You’re welcome to try. Your second Nature paper!
Richard,
What do you mean: that it wouldn’t be easy? If so, that would maybe suggest that the issue isn’t obvious and that addressing it will take some actual work?
Why, I’ve made no claims about the paper, one way or the other? (and, I think it’s already 2, but that’s neither here nor there).
That works fairly well, Willard. The writer is alluding to orthostatic micturition, which ze perhaps sees as a gender-hierarchical shibboleth imposed upon unquestioning male-identified students by the phallocentric patriarchy.
How about this one:
(Re your ‘should we care that most of Newton’s works have been written from the perspective of a mathematical mystic?’ – No, but… Manana, perhaps.)
> The writer is alluding to orthostatic micturition, which ze perhaps sees as a gender-hierarchical shibboleth imposed upon unquestioning male-identified students by the phallocentric patriarchy.
Not really, Vinny. It was more a study on how people who don’t identify with the good ol’ genders perceive restroom dynamics. There was a documentary part and an analytical part. The analytical part perused frameworks that look mundane in social and gender theory. Some work was going on in that abstract, whereas the topic of your pastiche and your “no, but” indicate that you’re just using my question to peddle your favorite meme.
Sure, it’d be possible to discern from the common contrarian crap some kind of grief of the white, aging, low middle class over the loss of their youth’s privileges. Perhaps Freedom Fighters fancy the freedom to frivolously feast over FUD. There’s an industry behind both. While I favor the second hypothesis, I’d rather blame the spirit of ClimateBall instead.
ATTP’s post was about Gender Studies and peer review. Are you saying that examples of peer-reviewed Gender Studies twaddle are mere off-topic meme peddlings if the person presenting them thinks they are twaddle but if someone else – you, perhaps – puts one forward as an exemplar of all that’s wonderful about Gender Studies then somehow it’s not peddling no more? Lawks, this philosophy stuff is mightily hard to pin down, guv’nor.
> ATTP’s post was about Gender Studies and peer review.
Not exactly, Vinny. I’d rather say that it was was about the authors’ dual inference:
That you and other contrarians revel in the publication of another hoax paper is only par for the ClimateBall course, but AT’s main point was that the second inference fell a bit short.
I, on the other hand, can be blunter:
https://twitter.com/nevaudit/status/867430151067402240
***
> Are you saying that examples of peer-reviewed Gender Studies twaddle are mere off-topic […]
No, Vinny. Only that your copy-pasting indicates you might need to read a bit more critical theory, i.e. what our two clueless Freedom Fighters misidentify as “academic fields like gender studies.”
It’s not like it’d be impossible to come up with constructive criticism of critical theory, e.g. Gadamer 1975. Vintage 1975. Fancy that. Joshua might like this one – it echoes many of his ideas. I don’t buy much of hermeneutics, and much less Kristeva’s crap, but gustibus and all that country music.
In any case, both the authors failed to respond to my tweets, including this last one:
https://twitter.com/nevaudit/status/868198560713592832
willard –
=={ Joshua might like this one – it echoes many of his ideas. }==
I’d have to be able to understand it first. Could you just give me a quick overview?
I have wondered if some of the worst gender studies studies are deliberately provocative, that the “evidence” that “confirms” them is not part of the study at all – or is in any way an academic process – but comes from the mining of the predictable responses, which will tend to see the misogynist ranters display their worst aspects in full view. Of course those are not broadly representative even if people with such attitudes are not figments of imagination.
Which isn’t the point in this case, even though the authors/conspirators sought to mine the predictable responses to the content – and to having it pass it’s “peer review” to publication – if only as a form of click bait, that raises it’s public profile.
> Could you just give me a quick overview?
Reading the conclusion one paragraph at a time ought to be enough.
This paragraph shows the difference between hermeneutics and critical theory:
In the following paragraph Gadamer should appeal to your sense of irony:
I rather like the paragraph that follows, since it shows a difference between the discourse of empirical science, which is based on monologues (or rather dialogues with nature), and interpretative disciplines (for lack of a better term), which are based on dialogues:
Contra Gadamer, I doubt that communication is impossible without a common ground and that critical theory fails reflexivity, but I agree with his overall conclusion.
Because, ClimateBall.
Thanks, willard. Indeed, there is a lot there that dovetails with my perspective.
I don’t know about impossible… That’s too philosophical for me. Maybe difficult. Also, I guess it depends on what you mean by communication.
“I, for one, would love to know who the two anonymous reviewers were on that follow up paper.”
You dont need to know WHO they are. You just need to know OTHER PAPERS that they either
Passed or Failed.
The reviewer DOI will then over time garner or lose reputation points. You dont even have to be invited to be a reviewer to do this.
Nice to see ya’ll discussing Gadmamer.
@wotts
“”Our advice was the opposite.”
What do you mean: that it wouldn’t be easy? If so, that would maybe suggest that the issue isn’t obvious and that addressing it will take some actual work?”
Our advice was that Nature would never publish a rebuttal of a paper on statistical grounds. Showing that Burke and co are wrong is trivial to anyone who understands stationarity and degrees of integration. We typically do this in third-year undergraduate economics, so it needs careful wording but is not particularly complicated.
Richard,
I don’t think the rebuttal has to be in Nature.
https://www.wsj.com/articles/the-campus-mob-came-for-meand-you-professor-could-be-next-1496187482?shareToken=stf41515773ad3490cb8140067a949560d
–snip–
Equality of outcome is a discredited concept, failing on both logical and historical grounds, as anyone knows who has studied the misery of the 20th century. It wouldn’t have withstood 20 minutes of reasoned discussion.
–snip–
Interesting. So equality of outcome as a concept is discredited and illogical. I hadn’t realized that. Good thing someone’s around to explain that so authoritatively.
Especially since it is so obvious that equality of opportunity is so logical and credible (not to mention established by history, as anyone who has studied 20th century misery would immediately recognize).
I always love me some reactionary claptrap.
Time for my favorite Anatole France quotation:
@wotts
“I don’t think the rebuttal has to be in Nature.”
Hmm. If you submit a “Nature published a silly paper” to an economics journal, the editor would desk reject with a “what’s new?”.
Richard,
Just to be clear, you’re complaining about influential papers that are wrong, but are not corrected. You’re aware of one, you think the error is pretty trivial, and yet you aren’t planning to do anything?
That isn’t necessarily what you need to do. If the paper is influential, that implies that result is of interest. One could always redo a similar study to produce an infuential, but correct, paper.
Steven,
I did see the Heterodox Academy article about the story, but I can’t read the WSJ version. Seems pretty unpleasant.
Richard wrote “If you submit a “Nature published a silly paper” to an economics journal, the editor would desk reject with a “what’s new?”. “
Indeed, writing a comment paper in an economics journal for a paper that appeared in an environmental science journal would be utterly unheard of!
The fact that the comment paper contained daft arguments (marginal assumption) and repeated use of meaningless/misleading “Null Ritual” hypothesis tests suggests the editor should have rejected it after review.
@wotts
This is not a trivial error. Their entire result collapses if the estimation is done competently.
Richard,
I meant, trivial to show that it is wrong, which is what you seemed to be suggesting (I don’t know if this is true, or not). Again, if there is an influential, but wrong, paper, and you know how to do the analysis properly, then maybe you should go ahead and do so. I’m not quite sure what you hope to achieve by going on and on about it in the comments of a blog post.
@wotts
Theorem
A stochastic process X cannot cause another stochastic process Y if X is I(n+m) and Y is I(n) for any n>=0 and m>0.
Proof
If X causes Y then Y is at least I(n+m). QED
Richard,
Why are you telling me this?
Anders – got a linky?
Joshua,
For what?
I had a quick look at the Burke, Hsiang and Miguel (Nature, 2015), it isn’t immediately obvious to me how Richard’s criticism applies (of course I am not an economist, but experience has shown repeatedly that I can’t take Richard’s word for it on statistical matters).
Dikran,
I think there is also some history here. There is a 2013 paper by Hsiang, Burke & Miguel and then a comment paper, which included Richard as an author, followed by a response by Hsiang, Burke & Miguel. Richard is now – I think – talking about a more recent paper by Burke, Hsiang & Miguel. Seems that this is an area in which there is still some debate.
ATTP, any reason you allow Tol to comment on that Nature paper, while he is at the same time repeatedly ignoring Dikran’s criticism?
Serious question.
Ah, seems that Richard is being somewhat “economical with the actualité”? The paper I looked at was this one. Sadly I suspect asking Richard to explain exactly how his criticism applies will result in the same sort of evasion that came of asking him to specify the penalty term he used in one of his studies (the closest he came was to say he used “a more appropriate Bayesian approach” which could have meant several things, and of course did not allow me to replicate his results), and then completely ignoring my other criticisms of his paper.
Anders – the article you mentioned to Steven.
Joshua,
Here it is.
Marco,
Honestly, because it’s probably easier (for me, at least) to let Richard comment than to somehow try and force him to respond to Dikran’s queries (not that it matters, but the last time I deleted a comment of his, he tweeted my university to complain). The irony of him complaining about errors in someone else’s paper, while ignoring those in his own, will – hopefully – not be lost on others.
FWIW I am not going to repeat them any further on this thread. I don’t really want to contribute to another thread being diverted onto a discussion of Richard (other than as an example of what we can reasonably expect peer-review to achieve).
Anders –
Thanks.
Wow, now I see that story’s all over the place, Zerohedgee, Tucker Carlson, Washington times, Washington Post, National Review, the American Conservative.
Even a two hour interview with Bret
“he tweeted my university to complain”
You’re letting yourself be bullied into submission.
P.S. Dikran: maybe you can write a short comment on Tol’s paper and put it PubPeer?
I guess that’s one way of looking at it.
Marco, I could do, but it probably isn’t worth the effort. Prof. Tol seems to have written a series of papers trying to argue that an analysis of the literature suggests there is some economic benefit to a modest degree of warming, when the only study that explicitly suggests a likely positive benefit (rather than being neutral) was his own. This reminds me of the old statistics quotes “he uses statistics like a drunk uses a lamp-post [or a sequence of lamp-posts] – more for support than illumination” and “torture the data enough, it will always confess”. The basic problem is that it is a small dataset with obvious potential outliers and variable/asymmetric uncertainties in the datapoints and if you keep re-analysing the same data again and again that is a recipe for conformation bias, especially if your initial attempts are full of “gremlins”.
@wotts
“Why are you telling me this?”
Earlier on, you wondered how complicated it would be to explain this. Well, it’s a two-line theorem with a one-line proof.
@others
There is indeed an earlier paper by Burke, Hsiang and Miguel that a lot of people took exception with.
For good measure, Hsiang is working on a paper that repeats the error of the earlier paper; and Burke and Hsiang are working on a paper that repeats the error of the later paper.
Richard,
I was more asking why your were telling me this, rather than trying to publish some kind of response. If it’s an influential paper, then surely you publishing something that is a correction to this would also be influential, and therefore worth your time?
Technically, you would also have to show that the authors you’re criticising actually did what you claim they did. Pointing out something that might be true, doesn’t suddenly mean that they made the error that you claim they made.
richard wrote:
“Theorem
A stochastic process X cannot cause another stochastic process Y if X is I(n+m) and Y is I(n) for any n>=0 and m>0.
Proof
If X causes Y then Y is at least I(n+m). QED”
Isn’t that just a more general restatement of the theory rather than a proof? I would have thought a proof would demonstrate that “If X causes Y then Y is at least I(n+m).” is true (or provide a reference). When having a discussion, it is generally a good idea to give some consideration to the knowledge held by the intended audience, at least if you want them to understand the point being made.
Marco Andrew Gelman’s WP article is a pretty good summary of the problems with the original. Richard’s outlier study was what was holding up the quadratic bulge in the original model; it is also very likely to be the thing that causes the breakpoint in the segmented regression analysis to be where it is. Plus ca changepoint, plus c’est la meme chose… ;o)
Note that in Tol’s figure 2 his own study lies well outside the “confidence interval” of the quadratic model once the gremlins are fixed and some additional datapoints added. The solution? Throw away the quadratic model and try something else (instead of throwing out the outlier)
@dikran.
If a stochastic process X is I(n+m) then the stochastic process aX is I(n+m).
Note that BHM assume that dY = aX + bX^2, with a,b>0, dY I(0), X I(1).
I wrote
but here is Richard assuming we all know what I(.) is (I looked it up), and being terse/cryptic. His new post is essentially just reiterating what he has already written without actually making a substantive addition. I also wrote:
Richard is clearly spinning this out as slowly as he possibly can, for instance by not specifying where (page/equation number) the authors “assume that dY = aX + bX^2, with a,b>0, dY I(0), X I(1).” etc.
Note there is also potentially a difference [sic] between Y and dY that in this context means it isn’t clearly an error (at least not to me as a non-specialist).
On the bright side, he has shown that he actually does see (at least some of) my posts, I was beginning to wonder… ;o)
Note also I wrote ” I would have thought a proof would demonstrate that “If X causes Y then Y is at least I(n+m).” is true (or provide a reference). “, but Richard has provided neither a proof, nor a reference (which I suspect is easily done).
@dikran
I indeed changed notation between Y and dY.
You do statistics, so I assumed that you knew that I(n) means Integrated of order n, or that the nth difference is stationary. Any textbook on time series will tell you this. Most people like Hamilton.
Richard,
This is a blog. I think Dikran’s point was that maybe you should consider that not everyone who reads your comments does statistics. Hence, you could try to elaborate a little so that those who are not expert understand what you’re saying.
Richard, not all statisticians work on time series. There are terms used in my field that I would not assume you (and more importantly the general reader of this blog) knew. So if my argument required e.g. reproducing kernel Hilbert spaces, and I actually wanted to make myself understood (rather than just make myself look clever), I would take the time to explain my point in terms likely to be accessible to my audience, rather than just say ” Any textbook on kernel methods will tell you this. Most people like Vapnik or Aronszajn.” I do know what I(n) means, as I said, I looked it up (just to be sure – I had a vague recollection from that last time I looked at time series analysis).
Richard writes:
“I indeed changed notation between Y and dY.”
given that ” I(n) means Integrated of order n, or that the nth difference is stationary.” that substitution couldn’t possibly cause any confusion! ;o)
I note you have not taken the opportunity to express your criticisms clearly, and are still; just leaving breadcrumbs for us to follow. Sorry, that is trolling, and I have better things to do than to indulge that sort of behaviour, especially as it is obstructing efforts to verify that your criticisms have some merit (which is precisely the opposite of what I would expect from a senior academic).
@dikran
My theorem was about any X and Y.
I changed notation to have X and Y as in Eq (1) in BHM. I should have alerted you, but you figured it out all by yourself.
This gave you the opportunity to lecture me, and not focus on the subject matter: The statistical nonsense that is BHM.
Richard,
Again, if this is an influential piece of work and the error is so fundamental, why aren’t writing some kind of response? It doesn’t even have to be a direct response, it could just be another paper that addresses the same issue, but does so correctly.
another breadcrumb (Eq 1 may be relevant).
“I should have alerted you, but you figured it out all by yourself.”
again this comes across as (rather unsubtle) showing off, rather than actually trying to communicate. It isn’t obvious to me that eq 1 involves I(0) and I(1) processes, so even though it is the only numbered equation in the paper, it isn’t clear that is the source of the apparent error.
“This gave you the opportunity to lecture me, and not focus on the subject matter: The statistical nonsense that is BHM.”
Nobody is lecturing you. I am trying to get you to explain the problem with the BHM paper in a way that I can understand your point (and have reasonable confidence that I have understood you correctly). I am trying to focus on the subject matter, but your unwillingness to do more than drop cryptic statistical hints is making that unnecessarily difficult. The fact that you continue to do so suggests that it is deliberate.
@dikran
Sorry. It’s Eq (15), p. 38, SI.
(The estimating equation is the first one I read, not the first one presented.)
another breadcrumb. Sorry Richard, I don’t have time for your games. It is almost as if you want to cast doubt on a paper, whilst at the same time making it as difficult as possible for criticism to be put under any scrutiny. I can see why you are not keen on following ATTPs suggestion and writing it up properly.
“(The estimating equation is the first one I read, not the first one presented.)”
my mind reading ability seems not to be functioning too well as I missed that completely! ;o)
Richard writes: “Sorry. It’s Eq (15), p. 38, SI.”
This is (almost) like getting blood from a stone, it shouldn’t take the exchange of this many comments before Richard finally specifies where there error in the paper can be found (I was going to count them, but life is too short).
The supplementary information doesn’t actually have a page 38. Did BHM have more than one nature paper in 2015? There is an equation 15 though, on page 8, but I don’t see any mention of I(0) and I(1) – I only skimmed the section though, there doesn’t seem much point in reading it in detail until Richard deigns to actually explain his criticism in a manner that I can understand.
Dikran,
Maybe Richard has invented 6 additional SI pages, a bit like inventing 300 additional abstracts? #FreeTheTol300!
LOL ;o)
I think I should just point out that earlier in the thread I wrote:
I am willing to entertain the idea that there is a problem with the BHM paper, but Richard’s behaviour gives the impression that he doesn’t want it evaluated.
Might be useful if Richard could clarify precisely which paper he is talking about, as it appears that it’s not quite clear.
SI has 62 pages: https://web.stanford.edu/~mburke/climate/BurkeHsiangMiguel2015_SI.pdf
Richard,
Actually, it seems that it does have 32 pages. On the Nature website, it starts from page 1. From your link, it starts from page 31. The equation you’re referring to would seem to be on the 8th page of the SI.
All nits have been picked, I hope.
Now, does the left hand side have the same order of integration as the variable of interest on the right hand side of the equation?
Richard,
If you would like to make a point, feel free to make one.
Peer-review is tainted, as is post-publication review.
Richard,
I get the impression that you are suggesting that other people should be doing more to address your concerns. If so, I’ve no idea why you think they should. If there is a major problem with a fundamental paper and you’ve noticed it, I still don’t get why you don’t simply try to write a paper that addresses the same issue, but does so correctly.
Richard wrote “All nits have been picked, I hope.”
This is not nitpicking Richard, specifying exactly where the error lay is something you should have done immediately, not after n comments.
“Now, does the left hand side have the same order of integration as the variable of interest on the right hand side of the equation?”
Richard, I have already said several times that I am not an expert on time series, so I don’t know. Perhaps rather than being patronising you could try explaining why the left hand side does not have the same order of integration as that on the right in terms non-specialists (like me) will understand. Of course if you just want to show off how clever/knowledgable you are, rather than communicate some of that knowledge, then then I’ll leave you to it.
@wottsywotts
Please shelve your concerns about my publication strategy, if only for argument’s sake.
I’m trying to convince you that there is something wrong with BHM15.
So far, I’ve failed to engage you. You have yet to say anything substantial on the estimation strategy of BHM15.
Note that I may be doing this in preparation for a rebuttal to BHM15.
Richard writes “Peer-review is tainted, as is post-publication review.”
Do you think refusing to give direct answers to technical queries about your work, or ignoring errors in your papers when pointed out or making criticisms of other peoples work and being unwilling to constructively explain/justify their criticism is beneficial in post-publication review?
Richard,
Well, yes, I realise.
This is largely because I can’t really see the point. I haven’t read the paper, that you brought up on this thread, so I don’t feel in a position to engage in any kind of serious discussion about it.
Indeed, because I don’t really have a view on it one way or the other. If you wanted to make an actual argument (rather than what seem to be soundbites) I’d be happy to read it.
If so, I look forward to seeing it.
Richard wrote “I’m trying to convince you that there is something wrong with BHM15.”
No, if that were the case it wouldn’t take a long exchange of comments just to get you to specify where you think the error in BHM15 is. If you wanted to convince you would provide an explanation of what is wrong and why, rather than just dropping a series of cryptic statistical breadcrumbs (often in inconsistent notation).
“So far, I’ve failed to engage you”
No, the fact that we are still trying to get you to explain exactly what the problem is, in terms we will understand, shows we are engaged, the problem is that you apparently won’t deign to do that. But don’t pretend the the lack of engagement lies with us.
@dikran
The problem is simple: BHM15 regress an I(0) variable on an I(1) one.
Richard,
That might be simple, but I don’t think it’s obvious to most readers of this blog (and not to me either). If you would like to elaborate, please do.
There are two steps:
First, does it make sense to regress an I(0) variable on an (1) one?
Second, is this what BHM15 do?
Richard,
There are many other steps. What is an I(0) variable? What is an I(1) variable? What do they represent? What is the calculation trying to determine? This is just a few of them.
An I(n) variable is integrated of order n. It’s nth difference is stationary.
The calculation is to determine whether there is a statistical association between the two variables, let’s say a correlation.
Richard,
I was more meaning that you try to explain this in a way that minimises the use of terminology. I don’t think that most people know what “intergrated of order n” or “stationary” mean.
Stationary means that the statistical characteristics do not change over time.
For instance, BHM15 take the first difference of the log of per capita income. If this is stationary, then the average growth rate of income per head is the same in the 1960s as it is in the 2000s.
The temperature, on the other hand, is not stationary: It is different, on average, in the 1960s and in the 2010s.
Richard,
Yes, I realise that.
Good.
So, can temperature, a trending variable, explain the growth rate of income, a non-trending variable?
This is Newtonian mechanics. Can a moving object exert a force on an object that is standing still? (No, Newton would say, unless there is a second, opposite force; or the second object starts moving too.)
I wrote “Perhaps rather than being patronising you could try explaining why the left hand side does not have the same order of integration as that on the right in terms non-specialists (like me) will understand. “ [emphasis mine]
Richard wrote
O.K., so you just want to be patronising and show off. Sorry, my patience is exhausted.
The emphasis should have been on the “why”.
Richard,
Are you suggesting that the growth rate of income is by definition a non-trending variable, or just happens to be non-trending over the time interval of interest?
The growth rate of per capita income is typically found to be non-trending, and this is a claim made by BHM15. It is in line with standard growth theory. Since the start of the industrial revolution, the economy has grown exponentially.
I have to head out, but I’ve just had a look at the abstract and that seems to indicate that they’re trying to relate economic activity and absolute temperature, not trying to find a relationship between economic activity and temperature changes. Maybe Richard can clarify.
@wotts
Read Equation (15).
Equation 15 in the SI is applies on a country-by-country basis (which is appropriate as climate change will have different impacts on different parts of the world). While it may be true that the growth rate of the global economy is considered to be stationary, I don’t think that applies to individual countries, for instance Greece, where there is clearly a long term trend (even without the recent crisis), or groups of countries, or perhaps even the global economy.
However, I am not an economist and hence may have misinterpreted either the paper or the data or both.
Right, well I think I have spotted one problem, which is that those figures may not be “per-capita”. However looking again at the paper, it says:
so firstly it is the first difference of the natural log of GDP per capita, and it also has been adjusted for inflation, so it isn’t as simple as Richard is suggesting (“Read Equation (15).” isn’t even a breadcrumb).
@dikran
Sorry. Economist deformation. Income is real unless stated otherwise. Logarithm is natural unless stated otherwise.
Richard, that is a breadcrumb that I found. Your reply adds precisely nothing to your explanation of the issue. It doesn’t even explain why economists use a “deformation”, or what “real” (as opposed to “imaginary”) income means (presumably inflation-adjusted), or what effects they may have on the assumptions of the model. You are just playing games again, so I shall re-issue my nolle-prosequi on this (as Jeeves would say).
@dirkan
Deformation refers to deformation of economists, rather than by economists.
In this context, the opposite of “real” is “nominal”.
Like wottsywotts, you are working hard to deflect from the two key questions:
1. Does it make sense to regress an I(0) variable on an I(1) one?
2. Is that what BHM15 do?
Richard,
Again, if you would really like to make some kind of substantive argument, go ahead. Just asking questions is not really likely to achieve very much. I’m not trying to deflect from anything. You introduced the topic; you get to substantiate your claims. I have no view, one way or the other, and little expectation that a discussion on a blog is somehow going to resolve this.
My questions
1. Does it make sense to regress an I(0) variable on an I(1) one?
2. Is that what BHM15 do?
My answers
1. No!
2. Yes!
Richard wrote “Deformation refers to deformation of economists, rather than by economists.
In this context, the opposite of “real” is “nominal”.”
again my mind reading skills appear to be defficient, one wonders exactly how I was supposed to know either of those things from “Economist deformation. Income is real unless stated otherwise”? Note saying ‘the opposite of “real” is “nominal”‘ does not in any way clarify what “real” means as “nominal” may well have some special meaning in economics that I am not aware of. Richard if you wanted to show that you were being deliberately cryptic, you could hardly do better. My intepretation is that it is a rhetorical ploy designed to make the workload of the discussion asymmetric. It is easy for you to make cryptic criticisms of a paper and to give repetitive, terse and uninformative replies. It is a lot more work for me to determine whether your criticism is valid (or even applicable) if you won’t meet me at least half way by using terms I am likely to be familiar with, or give direct answers to my questions.
“Like wottsywotts, you are working hard to deflect from the two key questions: ”
No, I was trying to get you to explain how those two “key” questions relate to the content of the paper in a form that I am likely to understand. The fact that you only respond with repetitive cryptic breadcrumbs, that don’t clarify anything much, rather than explaining, suggests that it is you that doesn’t want these two questions understood and discussed, and I don’t think anyone is fooled by your bluster.
@dikran
On the left-hand side of Equation (15) we have the first difference of the natural logarithm of per capita income (measured in real dollars per person per year).
BHM15 claim that this is stationary, a claim they do not test, but that is supported by earlier studies and theory.
On the right-hand side of Equation (15), we have temperature. The whole point of climate change is that temperature is not stationary. There is a debate in the literature on whether temperatures are trend-stationary or integrated of order one, but it has been warming.
Question 1
As portrayed above, is this a sensible equation to estimate?
Question 2
Is this an accurate representation of BHM15?
Richard,
I’m not sure that it is. I’ve only had a chance to look at their abstract, but it seems that they’re trying to find a relationship between economic activity and absolute temperature (not temperature changes, and not temperature anomalies). For example, their abstract says:
@wottsywotts
Don’t read the abstract. Read the equations.
Richard,
The abstract tells me something of the goal of the research, and something of what they concluded. It’s not clear that it’s quite what you seem to be suggesting.
Richard wrote “On the right-hand side of Equation (15), we have temperature.”
… and a whole bunch of other stuff, including country by country linear and quadratic functions of time.
“Question 1 As portrayed above, is this a sensible equation to estimate?”
Depends on the effects of the “other stuff”. For a regression model you need to consider the whole model and its estimation, not just one term (for example you could add a linear term to a model to allow the data-driven detrending of another variable).
“Question 2 Is this an accurate representation of BHM15?”
No, you didn’t mention the other stuff.
“Don’t read the abstract. Read the equations.”
Frankly that is a foolhardy approach to understanding somebody else’s work. The abstract and text of the paper is there for a reason, and if the equations are all you are interested, there is a very good chance to will fail to understand what they represent. To think otherwise is sheer hubris.
@dikran, wotts
I strongly disagree. If you want to know what someone did, you should look at their equations. If you want to know what they really did, you should look at their code.
Richard,
The point is that you need to know what the various terms in the equations represent. The equations, by themselves, rarely tell you this (unless it happens to be obvious).
@dikran
As to the “other stuff”, if your dependent variable is I(0), and one of your explanatory variables is I(1), what does that imply about either your error term or one of the other explanatory variables?
Russell and Joshua,
I edited your last comments, under the impression they were misplaced. So I can’t retrieve them. Feel free to repost them.
Willard- disappeared is not the same as ‘edited:
the upcolumn reference was to:
Joshua says:
May 31, 2017 at 3:49 am
I always love me some reactionary claptrap.
Please replace
Richard wrote “As to the “other stuff”, if your dependent variable is I(0), and one of your explanatory variables is I(1), what does that imply about either your error term or one of the other explanatory variables?”
See my earlier remarks about being patronising, especially when it is evading the point of the dependencies between the different parameters of the model (and treating them as if they were independent). You wrote:
“I’m trying to convince you that there is something wrong with BHM15.”
If you want to convince, it is you that needs to do the explaining; it is you that is criticising, so the onus is on you to justify the criticism. I am not interested in pandering to your attention seeking, so either give a clear explanation of the problem in terms your audience will understand, and justify the assumptions made (e.g. with verifiable references) or consider your audience to have diminished by one.
@dikran
Let me spell it out.
If the left-hand side is I(0), then the right-hand side is I(0).
There cannot be one I(1) variable on the right-hand side. There have to be at least two. In this case, as T cannot cointegrate with T^2, there are at least three I(1) variables on the right-hand side.
BHM15 therefore did not run a regression of income growth on temperature and temperature squared, but rather of income growth on the cointegrating vector of temperature, temperature squared and an unspecified third variable. The only candidate for that third variable are their year dummies, which they do not report.
In other words, they did not find an impact of temperature on economic growth, but an impact of the deviation of temperature from an unspecified trend. The claim in the abstract is spurious.
As the trend has a flexible form, it is no surprise that their results are highly significant. This is overfitting.
russell:
=={ the upcolumn reference was to: }==
I agree that extremism is extreme. I also think that the part of the quote immediately after you used your scalpel has some value:
Critical theory asserts that these key concepts of social life can be restored only by a critical discourse that unmasks the impact of interests – despotic, capitalistic, bureaucratic – on the formation of common convictions.
How bouts some critical discourse? Why did you cut the quote where you did?
> disappeared is not the same as ‘edited:
If I put what I have back, you should see that it is indeed the same. WP doesn’t have a control version system for comments.
As for your “but free speech,” please refer to AT’s post on free speech, and note that Freedom Fighters stuff should go into the relevant thread.
Things get lost- apologies for the transient indignation,
Perusing the Relevant Thread l found a source of recurring confusion– Excessive typeface modesty-
It’s really hard to note who’s authoring what, when under the boldface blackpost lede if the author’s name is in 10 point uncapitalized grey
So please, Willard don’t make like e.e.cummings
Understood, Russell. WP’s free CSSes are far from being pixel prfct. Sorry about that.
I said “See my earlier remarks about being patronising”
Richard said “Let me spell it out.”
I said “so either give a clear explanation of the problem in terms your audience will understand, …
Richard said “… but rather of income growth on the cointegrating vector of … “ (this is hardly “spelling it out”)
I said “… and justify the assumptions made (e.g. with verifiable references) …”
Richard said ” “ (i.e. nothing at all)
I said “… or consider your audience to have diminished by one.”
richards_audience–;
Sorry Richard, I am not impressed by your showing off, if you were willing to discuss the blatant (and elementary) statistical errors in your own work, I might have more patience with your criticisms of others.
Richard wrote: As the trend has a flexible form, it is no surprise that their results are highly significant. This is overfitting.”
like using a quadratic model to fit a dataset to accommodate an obvious outlier (which happens to be Richard’s earlier work), without which a linear model would probably suffice? ;o)
@ Richard Tol –
Hi Richard,
I’m a bit confused on your claims. How do we know that productivity growth rate is I(0)? I get that usually it is assumed to be I(0) in many economic studies, but maybe Burke et al. are assuming it’s non-stationary. Wouldn’t they need to do that in order to allow for the possibility that temperature affects productivity growth rate? Or am I missing something?
For equation 15, maybe they should have added energy prices. I suspect energy prices might affect economic growth, perhaps more than temperature.
Also, in the long run, shouldn’t the economic growth rates of all the countries converge in a sensible economic model? Because the way I see it, according to their model, one country could approach being infinity times richer than another country as time increases. So maybe they should do a pure time-series approach, which would obviously seriously reduce their number of observations and thus their confidence.
Interesting, haven’t really noticed these issues. I remember reading Burke et al. in 2015, and from what I recall they assume that the output effects of climate change are the same in the long run as in the short run. But this is clearly unrealistic; for example, if it started snowing in Kenya tomorrow, there would be significant negative impacts in Kenya due to the Kenyans not being used to snow. However, if Kenya had been receiving frequent snow over the past 50 years, the Kenyans would get used to the snow and adapt, lowering the negative economic impact. As a result, the study probably overstates the negative impact of climate change on economic output.
@-1=e^iπ
Omitted variable bias is not an issue in a model with saturated dummies.
RichardTol wrote “Omitted variable bias is not an issue in a model with saturated dummies.”
I wonder exactly what Richard means by “saturated dummies”? More breadcrumbs…
@dikran
The cointegrating vector of N I(1) variables is that linear combination that is I(0).
I’m sorry if you feel patronised. You claim to be a statistician. This is statistics. It is a different subdiscipline than yours, but I would expect senior lecturers to keep abreast of developments in cognate fields — if only to be able to advise students.
Richard,
Wow, that is quite a comment. Dripping in condescension. A not unsubtle attempt to highlight that Dikran is only a Senior Lecturer, while you’re a Professor. A rather insulting you claim to be a statistician. I think I would find it quite remarkable coming from anyone, but from someone as Gremlin-laden as yourself, it’s truly amazing.
I don’t want to get too personal, but why are you so afraid of having an actual discussion? Scared you might discover that you were wrong? Terrified that someone might shatter your world-view? Is it a form of imposter syndrome; worried that someone might question your credentials, so think that attack is the best form of defense?
I should probably add, that the thing Richard seems to have missed is that one way you can keep abreast of developments on cognate fields, is to talk with people in those fields, or with people who are aware of the developments in those fields. Normally such people are more than happy to clarify things to aid those who are trying to keep abreast of developments in cognate fields. Sometimes – as Richard has illustrated – they’re not.
@wotts
Note that the conversation is going a bit like this
0. Tol: Assertion
1. Dikran: Breadcrumbs
2. Tol: Explanation of term in above assertion
3. Dikran: Breadcrumbs
4. Return to 2.
My initial assertion was at a level that a third-year undergraduate in economics can understand. I am indeed not amused that someone who works as a statistician at a university does not have a rudimentary knowledge of one of the major developments in statistics in the last 30 years, a development that led to a Nobel Prize and is based on one of the most-cited papers in the field.
Richard,
Interesting that you chose to double down, rather than maybe re-evaluate your engagement. My own view is that you should probably be thankful that even if some people are unamused by some other people’s apparent lack of knowledge, most typically chose not to actually say so.
As I said above, I’m amazed that someone as Gremlin-laden as yourself feels in a position to be unamused by the supposed lack of knowledge of others. I, for example, have no idea if the thing that you think someone should know, is actually something someone should know. My own view is that fields have become so specialised that it is not unusual for some in a field to be unfamiliar with something that is regarded as a major development in another area in the same field. In my field, people are mostly happy to clarify these things if someone is unaware of a major development in an area outside their specific specialisation, but within their general field, rather than suggesting that others are lacking in some supposedly important piece of knowledge (maybe your field is different, or maybe it’s just you).
I think your description of the discussion is also rather nuanced. I couldn’t see much point in engaging in a discussion with you because I didn’t know enough about the example you introduced to have a view one way or the other, partly because I doubted that you would actually explain things clearly enough to help me gain some understanding, and partly because discussions with you are rarely worth the effort. I suspect that Dikran had similar views, but decided to have a go anyway. The outcome seems entirely predictable, and I doubt that Dikran is surprised that you’ve decided to descend to insults. I would have been much more surprised if this had turned out to have been a worthwhile discussion, than it turning out as it has.
Richard wrote “My initial assertion was at a level that a third-year undergraduate in economics can understand.”
the trouble is there is never a third year undergraduate economist when you need one! ;o)
It is ironic that Richard criticises my knowledge of statistics when he (i) treats zero as a positive number (ii) doesn’t consider the possibility of outliers in his studies (iii) uses unthinking null-ritual hypothesis tests (for large studies where common sense is enough to tell you that the two means will not be identical, so finding a statistically significant difference looks “sciency” but doesn’t actually mean anything) (iv) introduces obviously incorrect assumptions (“marginals”) because there isn’t enough data (in which case the validity of the assumptions is even more important as there isn’t enough data to over-ride them if they are incorrect), but worst of all (v) is unwilling to discuss these issues (and refuses to give a straight answer to a technical question about one of his papers).
“This is statistics. It is a different subdiscipline than yours, but I would expect senior lecturers to keep abreast of developments in cognate fields — if only to be able to advise students.”
If I had students that needed advice on time-series I would direct them to my colleagues who have expertise on the subject. Just as they would probably advise students to talk to me if they wanted to know about support vector machines etc. Hubris is not a good thing in an academic.
You will note Richard hasn’t specified what he means by “saturated dummies”. My guess would be one dummy variable for each value a discrete variable could take on (terminology is not used very consistently between machine learning, statistics and other “cognate fields”, so it could mean something else), however in that case it is difficult to see how that could prevent the possibility of omitted variable bias (the only cure for which is not to omit variables that happen to have some correlation with the variables you include).
@ Richard Tol –
Thanks for the response, but I’m a bit confused as to which paragraph you are referring to. I wrote 4 paragraphs in this comment section, the first of which was directed towards you.
The first paragraph was asking how we can assume that the left hand is I(0), which you did not answer, so I guess your comment wasn’t directed at the first paragraph (though I would appreciate if you could explain how we can assume the left hand is I(0)).
The fourth paragraph isn’t related to your comment so it can’t be that.
The second paragraph, maybe you were directing your comment at that? If so, I wasn’t trying to argue that the omission of energy prices in the model causes omitted variable bias. Rather that, from an economic point of view, it was interesting that they only looked at temperature’s effect on long term economic growth and not that of energy prices, even though both could have significant policy implications.
Or was your comment being directed at the the third paragraph? Are you arguing that the use of dummies means that the fact that their productivity functions do not converge in the long run for all countries is a non-issue?
@-1=e^iπ
You should not assume that a variable is I(0). You should test. BHM15 assumed, rather than tested. However, many other papers have tested this particular assumption on similar datasets, and could not reject the null.
BHM15 do not propose a model of economic growth. They only look at the impact of temperature. They kill the non-temperature pattern of economic growth with dummies. There would only be omitted variable bias if temperature were correlated with, say, energy prices.
ATTP writes “I think your description of the discussion is also rather nuanced.”
Indeed, Richard writes:
2. Tol: Explanation of term in above assertion”
Yes, Richard explained I(0) and I(1), …, in terms of the “cointegrating vector”, a term that I suspect you would only know if you already understood why I(0) and I(1) would be a problem!
ATTP wrote “The outcome seems entirely predictable, and I doubt that Dikran is surprised that you’ve decided to descend to insults. “
No, sadly I wasn’t greatly surprised.
ATTP wrote ” In my field, people are mostly happy to clarify these things if someone is unaware of a major development in an area outside their specific specialisation, but within their general field, rather than suggesting that others are lacking in some supposedly important piece of knowledge “
Exactly, the aim ought to be to communicate knowledge, rather than establish your place in the hierarchy.
Richard wrote “There would only be omitted variable bias if temperature were correlated with, say, energy prices.”
which seems a not unreasonable contingency… We have global warming, and energy prices seem to have been rising in the U.K.
Not clear what this has to do with “saturated dummies”.
How many tenured econometricians does it take to miss that Buhaug et al.’s first criticism was already addressed in HBM’s original article in a more stringent manner than your own counterexample, dear Richie?
@ Richard Tol – “BHM15 assumed, rather than tested.”
Where did they make this assumption? I can’t seem to find it in the Supplementary Information.
Hmmm.
“saturated dummies”
Irony stroked again?
@-1=e^iπ
The paper and SI indeed do not discuss stationarity. However, the replication code is here: https://purl.stanford.edu/wb587wt4560
As you can see, they use the “regress” code. They thus assume stationarity.
Richard Tol writes “BHM15 claim that this is stationary, a claim they do not test, but that is supported by earlier studies and theory.” [emphasis mine]
Richard Tol writes “The paper and SI indeed do not discuss stationarity.”
I think one of the problems may be that Richard’s readers pay rather more attention to what he writes than he does (and he perhaps should write more carefully if he is going to be so terse/cryptic).
Richard still hasn’t justified “Omitted variable bias is not an issue in a model with saturated dummies.”, which I don’t think is actually true.
@ Richard Tol –
Are you referring to the STATA code? Isn’t ‘regress’ just your standard linear regression?
So many .do files. Would be nice to know which one I’m supposed to look at.
Okay, so in GenerateFigure2Data.do, I found:
reg growthWDI interact#c.(c.temp##c.temp UDel_precip_popweight UDel_precip_popweight_2) _yi_* _y2_* i.year i.iso_id
Am I looking in the correct place or am I way off?
@-1=e^iπ
That’s the correct place.
I did not see a stationarity or cointegration test in any of their .do files.
@dikran
Fine. So write a letter to Nature protesting omitted variable bias in BHM15.
Richard,
Again, you’re the one doing the protesting, so maybe you should be the one writing letters?
Richard, no, I am asking you to justify your assertion that “Omitted variable bias is not an issue in a model with saturated dummies.”, which as I said, I don’t think is the case.
@ Richard Tol –
So now the claim has gone from they assumed stationarity for the dependent variable to they haven’t done stationarity or cointegration tests?
Have you tried doing these tests?
@ dikran, I think the claim is more along the lines of ‘in practice it isn’t generally an issue’.
more like “So now the claim has gone from they claimed stationarity to they assumed stationarity to they haven’t done stationarity or cointegration tests?” ;o)
@dikran
You’ll have to live with the assertion. This is not something we do in third year UG, but in second. You are clever enough to sort this out for yourself.
@-1=e^iπ
Yes, I have done these tests. As I said, I’m not particularly concerned about their left-hand side. It’s their right-hand side that worries me.
So if the dependent variable passes a stationarity test, does that mean that the estimates of the regression model are biased or inefficient or something? I’ve never encountered such a claim, but maybe I just went to a bad university & didn’t pay enough attention in econometrics class. I’m very confused as to what the issue is.
Richard,
You introduced this as a wrong, but influential, paper that had not been corrected. Now you’re suggesting that you’re worried about their right-hand side. I thought this was an example of something that was so obviously wrong, that it should be easy to demonstrate. Now it seems to be something that you worried about.
-1,
You’re not alone. This could be a feature, rather than a bug.
@ Richard Tol –
But I thought the claim was that they were assuming I(0) on left side then have I(1) on the right side. So that’s not the concern anymore?
Also as to your claims to what level of statistics is taught at each level at the UG level… it makes me think I went to a terrible university and that my degrees are useless. 😦
Richard wrote “You’ll have to live with the assertion.”
Richard, as far as I know, the only way to avoid omitted variable bias is not to omit relevant variables that might be correlated with any the variables you include in the model. I can’t see any reason why that should not apply to dummies. Sorry, I am not going to accept it on your authority, that isn’t how science works.
This is not something we do in third year UG, but in second. You are clever enough to sort this out for yourself.”
In that case, it should be no problem for you to give a verifiable reference to show that it is true.
In which year do you teach them that zero is a positive number?
@ dikran – maybe it’s that the dummies eat up most of the omitted variable bias, such that any remaining bias is likely to be minor?
I’m starting to think Richie’s the one who wrote Buhaug & al, for it seems to rest on a Gremlin-level reading comprehension:
Should I ask AndrewG to mansplain his fu to you, Richie?
@-1 It could be that “dummies” in econometrics are used in a way that I am unfamiliar with from machine learning. I have been looking into this possibility, but I haven’t found anything that clearly supports what Richard is saying. Of course it would be much easier if Richard would either (a) explain himself more clearly (and with some consideration for his audience) or (b) be willing to give straight answers to requests for clarification. However I think the asymmetry of effort in the discussion that Richard creates is a feature not a bug.
@-1=e^iπ
The problem is that the dependent variable is I(0) and the variable of interest I(1).
It seems that we’re back to where we started.
@wottsy
Indeed. None of you has answered my questions.
1. Is it a problem if the right-hand side is a higher order of integration than the left-hand side.
2. Is this what BHM15 did?
Instead, there has been all sorts of deflection.
@wotts
Returning to the topic of the thread, what we have seen is a high-profile journal publishing a paper that some would say is very dubious. We have agreed that there is no real recourse, as this journal is not known for publishing critique. We have also seen someone being proudly ignorant of basic developments in an adjacent field. None of this inspires confidence in peer-review.
Richard,
As I’ve said, on numerous occasions, I don’t know enough to answer your questions. If you would like to clearly lay out an explanation of the problem, using language that can be understood by those who might read it, feel free. If not, that’s also fine. I’m not convinced that even you believe what you’re suggesting.
No, we haven’t agreed on this (why do people trying to present an argument think they get to decide when other people agree with them?). I’ve even suggested a recourse. Someone who understands the problem (maybe you) could publish an analysis that is influential, and right. This is apparently important, so I don’t understand why you’re not doing so.
What we’ve actually also seen is someone so amazingly arrogant, that they can’t even be bothered to properly explain what they’re saying. This person also has a history of making rather silly mistakes, so that they can be so condescending is quite remarkable. It’s almost as if they’re too scared to present their arguments clearly, because if they did so, people would quickly see the errors in what they were suggesting.
I’m still confused as to how the left hand side is I(0)…
@-1=e^iπ
Good that you agree that the right hand side contains I(1) variables.
BHM15 take the first difference of the natural logarithm, operations that in similar datasets would render per capita income I(0). They use the Stata command “regress”, which is valid only if the dependent variable is I(0).
Richard wrote “Instead, there has been all sorts of deflection.”
No, there have been requests for clarification from people who are trying to understand the nature of your criticism, which is a normal part of rational discussion.
Richard wrote “We have agreed that there is no real recourse, as this journal is not known for publishing critique. “
Where exactly? Please give a URL. I may have missed something, but I can’t see anything that could be regarded as agreeing with that assertion.
Richard wrote “None of you has answered my questions.”
This isn’t a classroom, if you want us to be convinced you are right, it is you that should answer our questions.
The chutzpah of Richard of all people complaining that we aren’t answering hist questions is truly monumental!
Richard wrote “We have also seen someone being proudly ignorant of basic developments in an adjacent field. “
This is dishonest misrepresentation. Nobody is proud of their ignorance, it is just that some of us don’t let pride get in the way of learning from those who (apparently) know more that us. As I said, I would be surprised if most statisticians working on time series were conversant with the basic issues of (say) computational learning theory, what seems basic to one person is not necessarily basic to another.
Richard wrote “Returning to the topic of the thread, what we have seen is a high-profile journal publishing a paper that some would say is very dubious.”
You mean like “The Economic Effects of Climate Change” by Richard Tol? Or “Economic Impacts of Climate Change” by Richard Tol (apparently ” conditionally accepted for a learned journal.”, although maybe not a “high-profile” one) which has several clear errors that Richard is unwilling to discuss. Or “Quantifying the consensus on anthropogenic global warming in the scientific literature”, which contains an analysis based on an assumption that is obviously wrong and produces obviously incorrect conclusions #FreeTheTol300! (another question Richard doesn’t want to answer).
The first one though does appear to cast some doubt on post-publication peer review, given that Richard published a correction in 2014, which basically showed the conclusions of the original paper were incorrect, and yet (according to Google scholar) it is picking up citations faster now than it was prior to 2014. The 2009 paper has been cited 242 times since 2015, whereas the correction has only been cited 23 times. Ideally the publication of the correction should have resulted in people not citing the flawed study any longer (and maybe citing the correction instead – or at least both). I suspect that says something interesting about the research community. but I am not sure what (there are, very occasionally, highly cited papers in machine learning where the basic idea doesn’t actually work, so I don’t think it is just economics, but then again the societal impact of Richard’s paper suggests it ought to have had/be having more scrutiny).
FWIW I think Richard would do better to try and explain the problem in terms of the economics, rather than presenting it as a technical problem in the statistics (with no concession to the background of the audience). I’ve been thinking about the problem and I can see there are arguments for temperature or change in temperature affecting growth rate…
“They use the Stata command “regress”, which is valid only if the dependent variable is I(0).”
As far as I can tell, they used regress without any options, which is a standard linear regression. Are you saying that if the dependent variable of a standard linear regression is not I(0) then the regression is invalid?
@-1=e^iπ
Indeed: Regress returns the wrong standard errors in that case, severe downward bias leading to extreme overconfidence.
(And it’s not me who says this, this goes back to Yule.)
Sorry. p-values are wrong.
Rcihard wrote “And it’s not me who says this, this goes back to Yule.”
… but obviously far too difficult to give a reference that would make it easier for us to see Richard’s point. Just as he couldn’t/wouldn’t give a reference to the issue about saturated dummies (I suspect he was talking about fixed effects models?).
“p-values are wrong.”
I agree, we should use Bayesian posterior probabilities instead ;o)
Okay, I see now.
Forgive my ignorance. I think we barely covered this issue in econometrics and that was a while ago for me.
I would appreciate it if you could answer the following question (so that I will know what I should use if I encounter a similar problem in the future): What regression technique would you recommend for their equation 15 model if not OLS? Would Cochrane-Orcutt work?
Also, for attp and dikran, these links may be helpful:
https://stats.stackexchange.com/questions/94723/using-non-stationary-time-series-data-in-ols-regression
Click to access 10_cointegration_2pp.pdf
@dikran
Yule, JRSS, 1926
@-1=e^iπ
No, Cochrane-Orcutt would not work. Start with Dickey-Fuller and work forward.
Thank you Richard, looks like an interesting paper.
> Returning to the topic of the thread
The topic ain’t your next harassment target, Richie dear. That’s just what you peddled in the thread.
Shouldn’t equation 15 be cointegrated? In that case, couldn’t the equation be estimated using the Hendry-Sargan error correction model?
Sorry if this is a dumb question.
”
None of you has answered my questions…
…
Instead, there has been all sorts of deflection.
”
Oh dear.
This all seems somewhat familiar…
It’s instructive to see smart folks try (and try (and try (and try))) to give Richard Tol the benefit of doubt and take him seriously, but inevitably fail.
Amusingly, from the link TVRJH gives:
Richard Tol (@RichardTol) says:
May 9, 2016 at 11:02 am
Dikran: There is no error. There is an assumption.
How ironic!?
Of course that was different, …, we were criticising invalid assumptions in his paper ;o)
Here it is “proven” that the isobaric and the isochoric heat capacity of an ideal gas are equal. Published by an open access physics journal with open peer review, so you can even follow how this paper could be accepted.
http://www.sciencedomain.org/abstract/18839
Pehr, I very much like the way they provide a clear ‘audit’ trail for the paper. I am significantly less impressed how the Editors apparently allowed the author to completely ignore the highly critical comment from your compatriot. If he’s right, the paper is worthless.
Marco,
Indeed, it does seem like the kind of comment that should be addressed and, as I understand it, the reviewer is correct; the volume does not depend on temperature since it is simply the volume of the “box”.
Marco, ATTP, I agree with both of you. Moreover, Cv=Cp for ideal gases contradicts an overwhelming amount of experimental data and engineering experience. It also contradicts the first law of thermodynamics dU=dQ-pdV because when the gas is heated at constant pressure extra heat must be added due to the pressure-volume work of the gas on the surroundings.
At constant volume dQ=dU and Cv=dQ/dT=dU/dT.
At constant pressure dQ=dU+pdV=dU+RdT and Cp=dQ/dT=dU/dT+R=Cv+R because according to the kinetic gas theory U is only a function of T for an ideal gas.
It is amazing that the editors of a physical science journal don’t know such fundamental knowledge in physics.
@-1=e^iπ
There is various way to do this, Hendry-Sargan being one, but the left-hand side and the right-hand side have to be of the same order of integration — an assumption that is testable.
The problem with BHM15 is that, at first sight, their left-hand side is I(0) and their right-hand side is I(1) — and they surely do not formulate their model as an error correction one. They do not test their residuals for stationarity.
Richard,
Hold on, is this an influential, but wrong, paper (as you initially claimed), or an infuential, but imperfect, paper?
@ Richard Tol –
Thanks for the response.
Forgive my ignorance, but if you take the first difference of an I(0) series you end up with something that is stationary. So why wouldn’t Hendry-Sargan work if the left hand side is I(0)?
Also, couldn’t one claim that, even if the left hand side passes a stationary test, that it is an I(1) series that just happens to be sufficiently close to I(0) that it looks like I(0) and therefore Hendry-Sargan could work?
@-1=e^iπ
They could have used Hendry-Sargan, or a newer variant, to estimate the model. They did not, as you can see from Eq (15) and the .do files.
@wottsy
Influential but wrong.
Well, yes, I’m not surprised you’ve said this. I’m still to see a convincing argument that you’re right. If it is so influential and wrong, then maybe someone who realises this should be spending their time drafting some kind of response, rather than posting comments on a blog?
From the ReadMe:
The paper has been published in 2015.
Richie can’t blame Gremlins this time.
@wotts
“I’m still to see a convincing argument that you’re right.”
Let’s return to the Newton analogy. I(0) ~ standing still. I(1) ~ moving at a constant rate. I(2) ~ accelerating at a constant rate.
Can something be standing still and moving at the same time?
And no, the answer is not the Red Queen — you need to work with the same observer.
Richard,
According to Special Relativity, yes, but that’s probably not what you meant. What you’ve done in the above comment is extremely far from a coherent argument that illustrates why the paper you’re talking about is wrong. You can again, if you wish, but I have little hope that you’ll actually present anything convincing. To be clear, this is not me suggesting that it isn’t wrong, simply that I haven’t seen a convincing argument that it is.
@wotts
The case against BHM15 rests on two questions.
The first is whether something can stand still and move at the same time.
You appear to argue that it can, by which you have demolished my case against BHM15.
Richard,
I’m not quite sure why you’re finding this to difficult to understand (well, I’m pretty sure I know why) but I’m not really making any arguments, which I think I have pointed out more than once already. I don’t have a view, one way or the other.
As far as standing still, and moving, at the same time; well that depends on the frame from which you observe and whether or not the object is undergoing any acceleration. An object moving with a constant velocity can be moving in one inertial frame, while being stationary in another inertial frame. Since all inertial frames are equivalent, it can therefore be moving, and standing still, at the same time; it depends on the frame from which it is observed. I doubt, however, that this is relevant to this particular issue.
@wotts
If you observe the same object through the same intertial frame, can it stand still and move?
Richard,
No, in a single inertial frame, an object can’t both move and stand still at the same time.
@ Richard Tol – “moving at a constant rate. I(2) ~ accelerating at a constant rate.”
So since equation (15) has a time squared term, does that mean it’s I(2) and therefore Hendry-Sargan can’t be used? Or can it still be used if the series are co-integrated?
@ ATTP –
It’s not that hard to understand:
BHM15 try to estimate a model (equation 15) using OLS
One of the assumptions in OLS is violated due to the fact that the series is non-stationary
As a result, the usage of OLS is invalid and the nature of the error due to this violation of one of the assumptions is that the uncertainties of the estimates are significantly underestimated
This means that one of the main results of their paper (that they found a statistically significant negative effect of higher temperature on long term economic growth) may be invalid.
Richard has been asked more than once on this thread for evidence that growth rates are stationary. The best he has come up with so far is:
In the second sentence, Richard appears to be demonstrating a lack of understanding of null hypothesis statistical tests (not for the first time which led to the discovery that Richard doesn’t understand that two things can be similar but different [non-identical]). Not being able to reject the null hypothesis means that it is acceptable to assume growth rates are stationary, but it does not imply that they actually are stationary, or that you must assume they are stationary. It certainly is not justification for saying that BHM15 is incorrect for regressing and I(0) variable on an I(1) variable, because it doesn’t unequivocally establish that the I(0) variable is and I(0) variable.
In other words Richard has not demonstrated that the object in question is not moving, he has just asserted that it is not moving.
As an example, very few observable processes are exactly normally distributed (i.e. a Gaussian), because it is the limiting distribution as the number of random factors affecting the process tends to infinity. However statisticians frequently perform tests for normality, even though we know a-priori that the true distribution is only approximately normal, so if we gather enough data we will always be able to reject the null hypothesis of normality. So why do we perform the tests? Well frequently we use statistical methods that assume something is normally distributed, and we perform the test to determine whether the approximation by a Gaussian is likely to be reasonable.
@wotts
Thanks. You’ve now answered my question 1.
As read by me, Eq (15) of BHM15 does not make sense.
We can now move on to question 2: Do I read Eq (15) correctly?
Richard,
I have no idea. You’re claiming to have the expertise to make the argument. Make it.
@ dikran – “Richard has been asked more than once on this thread for evidence that growth rates are stationary.”
OLS assumes stationarity and therefore by using OLS, stationarity is assumed. Or at least that’s how I understand it.
“However statisticians frequently perform tests for normality, even though we know a-priori that the true distribution is only approximately normal”
As I understand it, Richard did a Dickey-Fuller test, or something similar, and it failed the test.
@wotts
You know my answer to my question 2: Yes, this is the only way to read Eq (15):
d ln y = T + T^2 + other stuff + u
I(0) = I(1) + I(1) + ? + I(0)
So ? must be I(1), and T + T^2 must cointegrate with other stuff.
“OLS assumes stationarity and therefore by using OLS, stationarity is assumed. Or at least that’s how I understand it.”
OLS assumes the residuals are normally distributed as well, but that is (almost) never actually true. While OLS is (as I understand it) non-optimal/biased in this case, that doesn’t mean the bias is sufficiently large to be problematic. It is ironic because Richard uses OLS in another study we have discussed here, where the assumptions of OLS are clearly violated.
As for Dickey-Fuller tests, I can only go by what Richard actually says, and since he spends most of the time trying to show off how clever he is, rather than explaining things clearly, it is unsurprising he makes such slow progress. Given that Richard has repeatedly demonstrated fundamental misunderstandings of very basic statistical concepts (such as NHSTs), I am not goung to take his word for it on more complicated issues.
“I(0) = I(1) + I(1) + ? + I(0)”
I would have thought a linear combination of non-stationary processes can be a (n approximately) stationary process.
@dikran
“I would have thought a linear combination of non-stationary processes can be a (n approximately) stationary process.”
That’s correct. That linear combination is called the cointegrating vector.
“That’s correct. That linear combination is called the cointegrating vector.”
So a stationary process can be “caused” by a combination of non-stationary processes (i.e. and I(0) process can be caused by a combination of I(1) processes)?
In the paper, the authors write:
Would you care to explain what that means, Richie?
@dikran
“So a stationary process can be “caused” by a combination of non-stationary processes (i.e. and I(0) process can be caused by a combination of I(1) processes)?”
Exactly.
But note that temperature and temperature squared cannot cointegrate.
“Would you care to explain what that means, Richie?”
The average slope of a parabola from x = a to x = b, where x = (a + b)/2 is the vertex, is zero.
> The average slope of a parabola from x = a to x = b, where x = (a + b)/2 is the vertex, is zero.
Thank you, Richie.
I don’t think the word “mean” means what you think it means. Or perhaps you do and are just pulling my leg. Which would not be a first.
So let’s try another bit:
What does it all mean, Richie?
@-1=e^iπ
“So since equation (15) has a time squared term, does that mean it’s I(2) and therefore Hendry-Sargan can’t be used? Or can it still be used if the series are co-integrated?”
Hendry-Sargan can be generalized to three or more levels of integration.
Before we return to Richie’s peddling, an update on the topic:
https://twitter.com/KetanJ0/status/872610517105352704
RichardTol wrote:
However Richard also (much earlier) wrote
Richard wrotes
“But note that temperature and temperature squared cannot cointegrate”
but temperature squared is not the only non-stationary process on the RHS of the equation (as I pointed out earlier in the discussion).
Richard’s contradiction here is because he is making a very poor job of explaining the problem, mostly because he is spending most of his time being patronising and showing off his knowledge of technical terms. AFAICS the problem is that OLS is biased/sub-optimal for estimating a model of this form, rather than the form of the model (as the above contradiction demonstrates). Of course, I could be wrong (not being an expert on time-series), but Richard’s obfuscatory approach to answering questions is likely to be a significant contributory factor.
@dikran
There are indeed three non-stationary variables on the right hand side.
The correct interpretation is that income growth (left hand side) is driven by the cointegrating vector (right hand side) rather than by any individual element of that cointegrating vector.
And what are the variables in the right-hand side of eq (15) again, Richie?
The “again” may not represent the possibility that you already identified them in the thread.
RichardTol is unable to admit his contradiction – news at 11. If the growth rate can be explained as a “cointegrating vector” including temperature, then temperature is a cause of the growth rate (I don’t think anyone is claiming that it is the cause), and the argument Richard has been droning on about for most of the thread doesn’t actually explain the problem with the paper (I agree the potentially is one – but Richard’s presentation of it is misleading, vague and a bit hyperbolic in terms of its significance AFAICS).
I vote we just use the log of temperature-squared.
@dikran
“If the growth rate can be explained [by] a “cointegrating vector” including temperature”
Sorry, you got that wrong.
The cointegrating vector explains, rather than its individual components.
The cointegrating vector consists of temperature, temperature squared, and a time trend that neutralizes the increase in temperature (squared).
If you want to extrapolate from the sample period to the future, then you should project not only the temperature but also the time trend.
@ dikran –
“rather than the form of the model”
I’d argue that there are problems with the form of the model as well. The model assumes that productivity for each country grows independently of all other countries. This is clearly nonsense as there are technological spillover effects between countries.
Richard wrote “The cointegrating vector explains, rather than its individual components.”
This is a bit like saying that only the weighted sum of attributes in a linear regression model explain the response, rather than the individual components, which is of course obvious nonsense. Temperature is a causal factor and Richard has contradicted himself, but can’t admit it, just like he is unable to admit the errors in his papers (such as treating zero as a positive number), plus ca change.
“If you want to extrapolate from the sample period to the future, then you should project not only the temperature but also the time trend.”
which is why the model used in the paper has a time trend components – duh! This is a straw man thrown in to deflect attention away from the contradiction. Sorry Richard, too transparent a ruse to be effective.
-1 possibly. All models are necessarily simplifications of reality, but some of them are useful, and I don’t have the domain expertise to judge whether that particular simplification is likely to be problematic.
@dikran
This is not a ruse. It is time for you to pick up your copy of Hamilton’s Time Series Analysis.
Richard,
I don’t think telling people what to do is all that helpful. Feature, not a bug, possibly?
@wotts
This is math.
I am not able to explain the equations to Dikran, so he should seek the advice of Jim Hamilton, who is generally regarded to be the world’s number one in explaining this.
Richard wrote: “This is not a ruse. It is time for you to pick up your copy of Hamilton’s Time Series Analysis.”
Translation: “It was a ruse, I contradicted myself and can’t admit it, and am forced to resort to patronising ad-hominems once more because I can’t form a cogent response”
Yawn, give it a rest Richard. Your repeated evasion on this blog has not gone un-noticed (e.g. from earlier in this thread), what makes you think it will work on this occasion any better than before?
Richard,
You introduced this as an example of tainted peer review. If you can’t explain it in a way that is clear to Dikran, then it’s almost certainly not clear to many others who might read this. If you can’t be bothered to explain it any more clearly, that’s fine, but it would be nice if you could own this; you can’t expect other people to do your homework for you (okay, you might be able to expect it, but believing that it will actually happen is a bit unrealistic).
@wotts
This is an admission of failure on my part.
Eq 15 of BHM15 reads
Growth rate of income = alpha * cointegrating vector
with
Cointegrating vector = beta * temperature + gamma * temperature^2 + delta trend
You cannot break apart the cointegrating vector as BHM15 do and Dikran wants to do.
This is obvious to me, but I’ve been steeped in this for 3 decades.
I am obviously not able to explain this.
Hamilton can explain.
Richard,
The problem, as I see it, is that peer-review is not perfect, and neither is any single paper. Finding an example of non-perfect peer-review, and a non-perfect paper, is, therefore, not really some indication of peer review being tainted (which was the topic of the post). If you find that you aren’t able to explain why this paper is so fundamentally flawed as to be an indication of tainted peer review, then that might simply indicate that it is more an example of people legitimately disagreeing about how to undertake some kind of analysis, than an example of a paper that is so fundamentally flawed that it brings into question a key aspect of how we assess research publications.
“This is obvious to me, but I’ve been steeped in this for 3 decades.”
The usual argument from authority.
“I am obviously not able to explain this.”
Indeed, as I pointed out earlier, the contradiction is due to Richard making a complete pigs ear of explaining the problem with the paper (which seems to be more to do with the estimation of the model, rather than its structure, and Richard going on about stationary and moving objects, whether I(1) can cause I(0) or not etc. merely obfuscating the problem).
Personally I find that if I can’t explain something, it probably means I don’t actually understand it very well (knowing the equations for a procedure is not the same thing as understanding it). There are lots of things I don’t understand as well as I would like, which is what makes research enjoyable.
> Hamilton can explain
I’m sure you can, Richie. Here’s thy Wiki:
https://en.m.wikipedia.org/wiki/Cointegration
Thy Wiki does not say what are the two l(1) series on both sides of the relationship in the example you are peddling, an example that might not be that relevant to the topic of this post, as AT’s suggests.
If you could clarify this, that would be great.
That you haven’t referred to Granger was a bit strange. It’s as if you never read the Auditor’s.
Richard writes:
Eqn 15 of BHM15 actually reads:
\Delta Y_{it} = h(T it ) + \lambda_1 P_{it} + \lambda_2 P_{it}^2 + \mu_i + \nu_t + \theta_it + \theta_{i2}t^2 + \epsilon_{it}
with
h(T_{it}) = \beta_1T_{it} + \beta_2T_{it}^2
It is almost as if Richard wanted to make himself as difficult to understand as possible!
Putting “$latex” before and “$” after an equation ought to work.
Like this:
Source: https://en.support.wordpress.com/latex/
@Willard – cheers!
It’s all Greek to me. But hey, the Tory’s have managed to bite themselves in their own arse and it’s just too hilarious for words. Trebles all round.
Sorry for OT, but this doesn’t happen every day.
Tories!
Effing ‘spell checker’.
@ Dikran – “I don’t have the domain expertise to judge whether that particular simplification is likely to be problematic.”
Oh come on, you don’t need a degree in economics to understand it, it’s a simple concept.
Do countries borrow technology from each other? Yes.
Does technology affect long term productivity? Yes, very much. The main reason people today are so much richer than those 100 years ago is due to technology.
Therefore, does it make sense to have a model where there is no interaction between countries and in the long term some countries obtain infinity times more productivity (i.e. become infinity times more advanced) than other countries? No. If you want to predict longterm productivity growth, you have to treat the planet as a whole because technology is shared between countries.
-1,
You need to read all of Dikran’s comment. It wasn’t disputing that countries do not grow independently; it was simply suggesting that it’s not necessarily obvious how such a simplification will influence the model being presented. Every model is a simplification of reality. Finding something that a model ignores, does not suddenly provide an immediate refutation of the model, because all models ignore something.
In addition to what ATTP wrote (with which I entirely agree):
“If you want to predict longterm productivity growth, you have to treat the planet as a whole because technology is shared between countries.”
No, sometimes if you want to make good predictions it is better to use a model that you know is “wrong”, but for which the parameters can be estimated reliably. For example spam filters typically use a “bag of words” language model (which treats documents as being assembled by picking words at random from a bag with fixed probabilities), which of course not how documents are written (except perhaps some of Richard’s blog comments ;o). However, if we use a more realistic language model, with all the interactions/dependencies between words, then there are so many parameters we can’t estimate them reliably from a practicable amount of data. Hence we get better predictions from an incorrect model with reasonable parameter estimates than we do from a more correct model with noisy parameter estimates.
The model in the paper already has plenty of parameters to estimate, so it is not clear that adding more interaction terms will make it give better predictions.
=={ The main reason people today are so much richer than those 100 years ago is due to technology. }==
Some might beg to differ:
“The main reason people today are so much richer than those 100 years ago is due to technology.”
I suspect a lot has to do with exploitation of resources (especially fossil fuels) – the industrial revolution wouldn’t have happened without coal.
@dikran
Wrong again. You show the equation as in BHM15, rather than as estimated.
And yes, rearranging terms matters. Y = aX + bW + cZ is not the same as Y = a'(X+b’W) + c’Z. These are not algebraic equations.
Richard says ” Eq 15 of BHM15 reads
Growth rate of income = alpha * cointegrating vector”
later Richard says
“Wrong again. You show the equation as in BHM15, rather than as estimated. “
I showed the equation as it appears in BHM15 because that is how it “reads” in that paper. I was illustrating why you are not able to explain the problem, which is because you are sloppy in your notation and you explanation of what you are doing – in this case saying “Eq 15 of BHM15 reads” when you were actually talking about how (15) of BHM15 is estimated. If you want to be understood, then try being precise and accurate (and consider the background of the audience).
“These are not algebraic equations.”
How is the reader of the blog expected to know that?
Lets see how many of these techniques has Richard used:
I make it four:
Trivial – (“third year undergraduates”)
Cumbersome notation (“not algebraic equations”)
Ghost reference (non-existent page numbers in SI)
Proof by authority (Richard’s own and Hamilton’s book)
If the model were being used to predict productivity changes over the next say 10 years, maybe it would be okay. But it’s being used to predict long run changes in productivity over 100 years. Ignoring all interaction between countries over such a long period of time is obviously utter nonsense; one just has to look at the past 100 years. If the USA were isolated and were completely to adopt all of the technologies an innovations from other countries over the past 100 years it would be far poorer and far more technologically behind than it is today.
Yeah, sure all models are wrong and are approximations of reality. But in this case it’s a giant glaring problem that completely undermines their entire productivity growth model.
How can you rearrange terms without presuming some algebraic properties, Richie?
@dikran
““These are not algebraic equations.”
How is the reader of the blog expected to know that?”
Well, the fact that this is the equation that was estimated was something of a giveaway.
I have no idea why this means that the readers of this blog are meant to know that these are not algebraic equations.
Nor me.
@wotts, dikran
Estimating equations hold only in expectation.
Expectation is a linear operator, which implies that in many cases you can manipulate these equations in the ordinary way.
However, by definition, a non-stationarity does not have an expectation (although it has, of course, a sample mean), so that you cannot treat a linear combination of stochastic variables as if it were a linear combination of non-stochastic variables.
The cointegrating vector does have an expectation.
You can therefore not take a cointegrating vector apart and move its components about.
> you cannot treat a linear combination of stochastic variables as if it were a linear combination of non-stochastic variables.
I’m not sure which variables you’re talking about, Richie. You still haven’t identified any. Also note that thy Wiki states:
https://en.wikipedia.org/wiki/Stationary_process
Furthermore, there seems to be some hierarchy in stationarity, e.g. weak stationarity. Perhaps clarifying this in terms of weather and climate would help. You’re our guru, Richie. Please do as you see fit.
Oh, and where’s the regress function in the authors’ code exactly?
@Willard – I mentioned that in a previous post:
Okay, so in GenerateFigure2Data.do, I found:
reg growthWDI interact#c.(c.temp##c.temp UDel_precip_popweight UDel_precip_popweight_2) _yi_* _y2_* i.year i.iso_id
reg is short for regress in STATA
Richard wrote “You can therefore not take a cointegrating vector apart and move its components about.”
but you can delete some of them (as you did)? I know you can’t do that, I am just pointing out how badly you are explaining the problem. I note you are still unable to admit your error of having written that is how equation 15 reads, when of course it doesn’t. I think I understand the nature of your criticism, but it is despite your explanations, rather than because of them.
BTW IFAICS nobody was suggesting taking a cointegrating vector apart and moving its components about, so that is just a straw man anyway.
-1 that is your opinion not a fact. The authors of the paper evidently don’t agree and they have actually performed some research on this. If you want to convince others, you need to do more that just make assertions.
As for technology. For several decades most basic technology gets written up in papers and patents and is there for all the world to see, so it is more or less a constant factor that can be bundled in with the dummies already in the model. IMHO – I am not an expert and don’t assume my intuitions are necessarily correct,
@dikran
“moving about” was an imperfect expression
you cannot take a cointegrating vector apart and separately interpret its components
> reg is short for regress in STATA
Thanks again, Richie.
Intriguingly, there’s also a file called cgmreg.do.
Now, what are the variables, and to what are they referring?
***
> you cannot take a cointegrating vector apart and separately interpret its components
Of course you can, but when you do, some (statistical and algebraic) properties fail. Which ones again and when, Richie?
Richard wrote ““moving about” was an imperfect expression”
Your explanation of the problem with the paper has been filled with such “imperfect expressions”, perhaps you ought to try being less cryptic, less patronising and consider the background of your reader. If only someone had pointed that out to you earlier in the discussion, you might have been able to put more effort into communicating your point more clearly ;o)
And to answer Wottsy’s questions:
This has been written up, and is currently under review at a learned journal.
Burke and Hsiang have been told, by me and others, that they got it wrong. Not sure whether anyone has informed Miguel.
Richard,
Good, I look forward to it. Did you do this between the time I first suggested you should respond and now, or did you just decide not to point out that you had already submitted a response? In fact, it’s slightly odd as your initial responses suggested that you couldn’t be bothered.
They may well be wrong, but – as you should well know – a group of people telling others that they got it wrong doesn’t immediately make it so. Sometimes it takes time to convince people that they were indeed wrong. Sometimes those who assert that others are wrong, are mistaken. In a sense it’s one of the strengths of the scientific process; there aren’t really arbitrers of who is right and who is wrong. It’s one reason why an understanding of a consensus can be important; it gives an insight into who is probably wrong, and who isn’t. It too isn’t perfect, but it is still a useful indicator.