Thompson, Erica (24 November 2022). Escape from model land: how mathematical models can lead us astray and what we can do about it. London, United Kingdom: Basic Books. ISBN 978‑1529364873. Hardcover edition. 
This beautifully written book confronts a number of uncomfortable issues usually left untouched and unexamined by those building and deploying numerical models in support of public policy.
The term “model land” is used by Thompson to describe the idealized realm these models inhabit: internally consistent, coherent, and possibly elegant, necessarily limited in scope, detail, and diversity, and also isolated from exogenous influences and effects, beneficial or otherwise. We openmoders are all card‑carrying members of our own model lands.
Thompson argues that the bridge between a model land and the real world is as important as the quality of the numerical models being constructed. Her book focuses on scientific models that cross‑cut public policy and thereby enter the political sphere — in her words, modeling to “inform decision‑making” rather than “scientific understanding” (p183). Climate models, infectious disease propagation models, and financial sector models are mostly used to underscore the themes presented. Scenario‑based modeling is barely mentioned and energy system models not at all.1
Model outputs that seem to align with real world observations without a considered evaluation of model performance are duly described as “statements of faith” (p7–8) (is that an oblique reference to artificial intelligence?). Moreover, one “cannot avoid model land by working ‘only with data’” (p7).
Thompson rightfully observes that most numerical modelers derive from a narrow slice of society in the global north and invariably hold advanced university degrees. Thompson uses terms like “standpoint bias” (p10), “privilege hazard” (p189), “symbolic violence” (p191), and even “manipulation and coercion” (p200) to underscore this situation and the potential jeopardies that may arise.
The notions of open source, open data, and open science are not traversed in this book at all. The closest Thompson comes to these concepts is this remark regarding infectious disease spread and control software (p198):
I do not know, but if the covid experience encourages some of those modelers to tidy up their domain and make make their models publicly accessible, it might be a good thing.
The experience referred to above (p191, 197–198) was the public outcry that the C language codebase primarily written by UK epidemiologist Neil Ferguson be released for inspection and confirmation. That pressure led to the Ferguson codebase being placed on GitHub and refactored in C++ as CovidSim by interested members of the public with support from tech companies.2,3,4 CovidSim is licensed GPL‑3.0‑or‑later. The earlier closed results, which the UK government had used to inform their undeniably harsh covid‑19 lockdowns were, fortuitously, broadly replicated by those new efforts.
Thompson also discusses at length the mathematical treatment of risk and uncertainty in numerical models. While astute, this material is not especially relevant to scenario‑based deterministic modeling paradigms — as used extensively within this modeling community.
The circa 1992 onward DICE model from William Nordhaus attracts coverage — and in particular his conclusion that 4°C of global warming should represent an optimal trade‑off between mitigation effort and planetary damage (p166–167). While that result is clearly nonsensical by today’s standards, Nordhaus, to his credit, did make his GAMS code freely available for other researchers and the interested public to examine and experiment with.
The DICE model leads on nicely to the notion that all numerical models embed values separate from facts — as Thompson duly opines (p76):
If we can mostly agree about what we think are facts (even if we are both wrong), then we can have a genuine discussion about values.
Indeed, this matter of embedded values should be tackled head‑on because “models always contain values” (p200). Moreover, modelers should make their value judgments explicit (p220):
All models require value judgments. If you cannot find any in your model, look harder or ask someone else to look for you, preferably someone with a non‑technical background who is directly affected by the decisions informed by your model. They will find some.
The book does not traverse the idea of model‑mediated public consultation however — an idea now being experimented with by energy system modelers in the United States and possibly elsewhere.5
In the final chapter, Thompson lists five principles to improve model development and deployment (p219–222) — to which I would suggest a further two principles:
6. Embrace open science and build overarching communities to tackle domain‑wide issues — such as shared standards and community‑based data curation
Indeed, much of this is what the Open Energy Modelling Initiative (openmod) was established to pursue. There are now high level projects covering semantics (Open Energy Ontology), metadata standards and cataloging (Open Energy Databus), and logical and physical data infrastructure (Open Energy Platform), together with broader work on research data management, model comparison exercises, and analytical repeatability. Most of these efforts are not legally feasible without suitable public licensing in place.
7. Partner, formally or informally, with analysts in the global south
The adoption of open source principles by a number of energy systems projects has allowed bridges to form with analysts in the African continent, Latin America, Asia, and elsewhere. Experience to date has shown this interchange to be organic and fluid — and while adequate resourcing is necessary for effective collaboration, institutional inertia and lock‑in can be readily side‑stepped.
To close, I would encourage Thompson to write an addendum which presents open science in this context as non‑negotiable and north/south partnerships as valuable and effective. Open science enables transparency, which I regard as a necessary prerequisite for meaningful engagement, and can thus help address some of the diversity deficits that Thompson so clearly articulates. And open science enables north/south collaborations to form naturally, allowing soft technology transfer in both directions while simultaneously respecting sovereignty. There are a number of recent examples of this dynamic in play.6
Thompson’s book is highly recommended for anyone interested in the self‑evidently tense intersection between numerics, systems, public policy support, and political decision‑making.
Notes
-
I take the reference on p159 to refer to integrated assessment models.
-
My remarks also based on recent conversions with a specialist who provides IT services research support at Oxford University.
-
Fund, John (6 May 2020). ‘Professor Lockdown’ modeler resigns in disgrace. National Review. USA. Quote: Last March, Ferguson admitted that his Imperial College model of the COVID‑19 disease was based on undocumented, 13‑year-old computer code that was intended to be used for a feared influenza pandemic, rather than a coronavirus. Ferguson declined to release his original code so other scientists could check his results. He only released a heavily revised set of code last week, after a six‑week delay.
-
CovidSim should not to be confused with CovidSIM developed by German researchers.
-
See for instance: van Bruggen, Anne, Igor Nikolic, and Jan Kwakkel (January 2019). “Modeling with stakeholders for transformative change”. Sustainability. 11 (3): 825. ISSN 2071‑1050. doi:10.3390/su11030825.
-
This list of publications for Africa provides some context. Other regions are covered on that same site.
▢