Models and scenarios

I was following, or trying to, a Twitter discussion about models and scenarios. It was – I think – about models that forescast technology development, and you can find it here if you’re interested. I didn’t entirely follow it, but my impression was that the suggestion was that if the scenario on which you were basing your model was unrealistic, then what you infer from your model could be very wrong. For example, if your baseline assumptions don’t properly reflect current policy, then you might infer a greater benefit to some action than is actually likely.

What I didn’t quite get is the reason for this. In many physical models, you can still infer something about how the system will respond to some perturbation, even if the underlying model does not capture the full complexity of the system. Of course, there are limits to this, but it is quite common to use relatively simple models to try and understand how a physical system will evolve.

So, is the problem with these forecast models that the system is so sensitive to the underlying conditions that if these don’t properly represent our current conditions that you really can’t say much about how the system will respond to changes? In other words, is it related to the lack of structural constancy that Jonathan Koomey discusses in this paper.

Alternatively, is it that people are not being clear about the limitations of their analyses? For example, we can’t use climate models to forecast the weather many years into the future, but we can use them to say something about how the climate will probably change if we perturb the atmospheric CO2 concentration by some amount.

I don’t actually know where I’m going with this. I didn’t completely follow the discussion and couldn’t quite tell what the actual problem was. It did seem, though, that the suggestion was that the underlying scenarios had to properly represent our current policy landscape and I found that slightly surprising. I’m much more used to the idea that one can use simple models to try and understand how a system responds to changes, without requiring that the model fully represents the complexity of the system being considered.

My concern would be that if the model is extremely sensitive to the underlying scenarios, then it would seem very difficult to be confident in the model results. As I’ve already said, though, I may have misunderstood what was being suggested, so would be pleased to have this clarified.

This entry was posted in economics, Research, Science and tagged , , , . Bookmark the permalink.

14 Responses to Models and scenarios

  1. RickA says:

    Sounds like a garbage in garbage out situtation.

  2. Everett F Sargent says:

    Did you get this far in that discussion?

    IEA’s climate models criticised as too fossil-fuel friendly

    So if the IEA only included a 1.5C scenario, which, you know, is the only realistic scenario going forwards (from a long term climate change aspect), even though we are not now today following even a 3.0C scenario (IMHO), the IEA is damned if they do and damned if they don’t?

    (I have all their WEO’s excluding 2017 (found 2018 about a week ago), still looking for that free one)

    That this discussion happens on an annual basis is only a cowinkidink. :/

  3. Everett F Sargent says:

    I’ve committed the Earth to a -1.5C scenario, on paper, in the year 2525. if man is still alive, if woman can survive, they may find ….

    See how easy that was? All you have to do is write it down on FF free paper. 😉

  4. It may be difficult to model broad-based innovation trends, but it is not at all difficult to estimate progress in certain specific technology cases. Ray Kurzweil has been doing it for decades and I actually did some of it myself to model my projected expansion of solar power. (I thought I was being too optimistic. In fact I underestimated its growth by about 10%.)

    By the way, the result of my modest efforts was a recommendation to turn our attention to rehabilitating the electricity grid, as it will be a real constraint to growth starting around 2033. Yes, we need batteries and improvement in batteries. But when solar (and wind) start to get around 30% of energy, we’re going to need a better grid.

  5. Everett F Sargent says:

    Note to self: All the WEO’s through 2016 are now free on their website (that happened about a year ago, before then, it was only through virulent searching on the internets).

    Also, this is the current latest BGR energy study, circa 2018 (in German, the EN version is always a few months later) …

    Please note their Kohle resource estimates versus their Erdöl and Erdgas estimates.

    Maybe BGR should not be ‘so called’ optimistic on their Kohle resource estimates? Justin Ritchie, PhD thinks so.
    Why do climate change scenarios return to coal? (journal paper)
    The 1000 GtC Coal Question (journal paper)

    The ‘we will run out of FF’s by 2100 gambit’ includes Kohle. Of course, now that fracking is a global endeavor, I’d expect natural gas resources to only go up. :/

  6. Everett F Sargent says:

    My rant/screed/manifesto bought and paid for by … wait for it … ExxonMobil …

    If you believe that one then you will believe anything. 😦

  7. Everett F Sargent says:

    If I’m not mistaken, the original discussion revolved around the use of IAM’s …

  8. Everett F Sargent says:

    OK, so clicking on that tweet (above) brings up parts 1, 2, 4, 5 and 6 (of six parts) AFAIK. This tweet shows part 3/6 …

  9. Everett F Sargent says:

    This is part 3/6 which is missing from the above …

    (last post vanished, perhaps too many of me in a row)

  10. Everett F Sargent says:

    Ouch, it showed up, please delete last me post and this one. TIA

  11. Everett F Sargent says:

    I’ll shorten this up a bit (e. g. shut up). The paper ‘Macroeconomic impact of stranded fossil fuel assets’ coauthored by Hector Pollitt (available for download here) …
    is also an IAM, just like the IEA models. One can of course be hopeful and plot a 1.5C trajectory assuming divestment, stranded assets and continued technology improvements. These collectively are what is known as assumptions that go into any IAM. These should be collectively be known as Integrated ASSUMPTION (not assessment) models IMHO.


  12. angech says:

    “if the scenario on which you were basing your model was unrealistic, then what you infer from your model could be very wrong.”
    The inferences would still be right for the model used.
    The problem is using an unrealistic model should [not always] give an unrealistic inference.
    It should only be very wrong if it was very unrealistic.
    you can still infer something about how the system will respond to some perturbation seems fine as a comment.

  13. Can this be boiled down to:
    The quality one’s answer is dependent on the quality of one’s questions.

  14. The thing is that physical models are testable against history in a way that constrains the spread of possible model structures… and physical laws stay constant over time. That makes results from physical models more consistently useful, even if aspects of the model are not right.

    In contrast, economic models have to make assumptions about future technology costs and BAU policies, which can’t be constrained well by historical patterns. It wouldn’t be hard to make a reasonable model that reaches zero carbon emissions in 2050 even in the business as usual case… thereby making climate policy unnecessary.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.