Code
There is no barometer as such for open modeling that I know of. The Open Energy Platform listing of energy system modeling projects notes 97 with open source software licenses from a total of 219 such projects. So that represents about 45% as a first‑pass metric.
Any barometer would need to distinguish between:
- unlicensed software made public but with no legal right to compile, interpret, and/or run
- the act of simply “throwing one’s code over the wall” albeit with an approved open source license
- projects committed to genuine open source development and trying to build enduring user/developer communities
The “throw it over the wall” concept (philipashlock 2015) refers to projects that technically publish their codebase with an open license but do nothing more — indeed, rarely if ever does anyone clone the repo to see if the source code even builds or runs. And there are usually no issues filed or fixed either for that matter. (I know of a couple of examples in the energy domain, but will refrain from providing names.)
The openmod is, of course, focused on the last of those three strategies listed above.
An allied theme is a recent US Federal Government requirement for procured software, both explicit and embedded, to be shipped with a software bill of materials (SBOM). More here: https://www.cisa.gov/sbom. I presume that requirement would apply equally to energy system models purchased for in‑house usage too. In addition, automated dependency auditing software normally screens for legal compliance too (and the underlying component graph may approach 20 000 nodes).
Data
Returning to the European Commission open science monitor (see link in the original posting). Many of the official definitions for open science regrettably fall well short of what is legally required to rerun the numerics and republish the original inputs and your generated outputs — in short, one cannot lawfully replicate previous scientific work. It is essential that the code is released under an OSI‑approved software license and the data under Creative Commons CC‑BY‑4.0, CC0‑1.0, or something inbound compatible. The European Commission site just quoted falls well short of those requirements — for instance, this definition for open research data is completely enfeebled (Anon 2019, European Commission no date) and entirely unsuitable for reproducible science in any genuine sense:
Open research data refers to the data underpinning scientific research results that has no restrictions on its access, enabling anyone to access it.
The French open science monitor (cited in the original posting too) similarly falls back to open access — the same threshold used by the European Commission. Access does not require any form of license — and if there is no suitable data‑capable open license provided, then republishing that data in original or modified form carries an extant legal risk. Under some circumstances, such republication can even class as criminal (and there is one such current case in Germany which I will blog about in due course).
Schwanitz et al (2022) screen 80 databases useful for seeking low carbon energy transitions against the FAIR data principles. While noting that legal reusability is not one of the included principles — FAIR merely requires that the licensing terms be clear and accessible (condition R1.1) but is entirely silent on what an appropriate legal grant to use and reuse might comprise. In particular, the authors tested in detail how machine‑friendly the selected databases actually are. The figure below summarizes their findings regarding the machine‑actionability of metadata and data identifiers in the context of FAIR. A relatively poor showing, I would say.
Figure 3 from Schwanitz et al (2022) showing FAIR compliance for 80 databases with the X0‑style identifiers mapping back to the various FAIR principles (CC‑BY‑4.0 license, further credits embedded in the PNG file).
Closure
If you want my impression and erring on generous: about ten percent of the numerical models used in Europe for developing official public energy policy carry suitable open source licenses — and about thirty percent of the datasets needed for such analysis carry suitable data‑capable open licenses. Again, a poor showing for a region that typically prides itself on its advanced level of democracy.
Even the PAC project, supported by a federation of European environmental NGOs, has opted for proprietary software to undertake the necessary systems analysis. Environmental NGOs appear not to prioritize transparency in this context — indeed after many attempts by myself to engage established environmental NGOs on these matters, I have come to the conclusion that most do not remotely understand the issues involved.
The original question of how closed and open energy system models might compare in terms of sophistication and uptake is difficult to assess for obvious reasons. I would guess that the good closed models will be no less sophisticated than the good open models. That is about all one can say without a publicly funded model comparison. But that is not really the point — we all know that numerical models can be selected and tweaked to support preferred outcomes. And that the only known antidote to such influence, be it latent or intentional, is open science and independent challenge.
The legal interoperability of data and datasets needs much greater attention from researchers and policy makers. The free software world has invested heavily in mostly resolving this question for software and a similar effort is required for information.
There is a very long way to go to arrive at genuinely open science and genuinely open public policy analysis — at least in Europe and the United Kingdom. On that final note, the analysis recently used by the UK government to form its net‑zero policy is currently subject to three separate judicial review actions by Friends of the Earth, ClientEarth, and the Good Law Project respectively in the High Court of Justice (Vaughan 2022).
References
Anon (4 April 2019). Open science monitor — updated methodological note. Brussels, Belgium. Marked “Subcontractor: Elsevier”. 
European Commission (no date). Open science monitor. European Commission. Website. Accessed 1 June 2022.
philipashlock (2015). Releasing open source. Civic Commons.
Schwanitz, Valeria Jana, August Wierling, Mehmet Efe Biresselioglu, Massimo Celino, Muhittin Hakan Demir, Maria Bałazińska, Mariusz Kruczek, Manfred Paier, and Demet Suna (25 March 2022). “Current state and call for action to accomplish findability, accessibility, interoperability, and reusability of low carbon energy data”. Scientific Reports. 12 (1): 5208. ISSN 2045‑2322. doi:10.1038/s41598-022-08774-0. CC‑BY‑4.0 license.
Vaughan, Adam (1 June 2022). “UK government faces legal challenges over net-zero climate plan”. New Scientist. ISSN 0262‑4079. 
