UK Government's Home Energy Model - Any Thoughts?

Hello All

Has anyone looked at the UK Government’s Home Energy Model that is planned to replace the current Standard Assessment Procedure (SAP)? They have published a consultation document here: Home Energy Model: replacement for the Standard Assessment Procedure (SAP) - GOV.UK

The open source Python code is available at: https://dev.azure.com/BreGroup/Home%20Energy%20Model

The model is at an early stage, and they are asking for feedback (by 6 March). I would be interested to know what others think and what experience people have of using it.

My current thoughts are:

  1. It’s open-source - good!
  2. The idea of making the Python code itself the legal reference for the methodology is under consideration. It would replace the 200+ page SAP specification. I’m sceptical about that

I would be interested to know what others think and what experience people have of using it.

Regards
Chris Gordon-Smith

A few thoughts from a distance. Noting that I have only briefly skimmed the official material, but did manage to read the executive summary. For clarity, the “Home” in the project title refers to an official methodology for “assessing the energy performance of dwellings” in the United Kingdom (that quote from the code repo preamble linked in the second URL above).

Reliance on a simulation‑based assessment methodology in this regulatory context seems generally workable and appropriate to me. So I am less skeptical than @Chris.Gordon-Smith on the general count of implementing regulations in code rather than through sets of prescribed calculations and accompanying technical standards.

The Home Energy Model methodology is being considered for use in binding legislation as well (consultation document, §3.1) and I presume that is the ultimate goal:

We are currently considering whether the open-source code could serve as the approved methodology for regulatory uses of the Home Energy Model (e.g. for Building Regulations compliance purposes). This would mean that the code replaces the specification document that has served this role for previous versions of SAP, and act as the ultimate legal reference for the methodology.

Stepping back, the concept of allowing building developers to submit their own thermal performance analysis for approval using simulation methods is not new. What is novel (as far as I am aware) is to offer an officially maintained simulation environment to facilitate that process and, moreover, to constitute its development as open source.

The choice of an open‑source MIT license is clearly useful. If not already covered in the GOV.UK plans, there needs to be an equal commitment to placing the calibration data and related information under Creative Commons CC‑BY‑4.0 licenses. Indeed, good software engineering suggests that the framework (code), the calibration data (empirical), sought energetic performance (prevailing and future policy criteria), and the instance data (often speculative) be kept separate, made public as appropriate, and suitably licensed per type.

GOV.UK suggests the methodology is intended to serve a number of roles and should accord users flexibility by separating the building physics from other more discretionary numerical and contextual aspects.

There is also to be some kind of ongoing quality process to confirm and improve the methodology and its applications. Questions arise as to who will determine which pull requests are accepted and how this shifting development will impact on the official status of the codebase/datasets? And, in particular, what will be the independence of the codebase maintainers? Stepping back, there are clearly some new statutory issues to traverse when adopting simulation‑based standards.

Here is the right place to make an oblique reference to the ongoing Horizon Post Office scandal in which important software was deemed “correct” by legal default.

Regarding the MIT license, python is not a compiled language, so third‑parties making forks will still need to distribute source as the means of circulating their modifications. So a permissive software license is appropriate, in my view. And as indicated earlier, the idea of forking is somewhat counter to the idea of having a single reference implementation to support statutory purposes.

The consultation documents are issued under an OGL‑UK‑3.0 license. Despite official claims to the contrary, I do not believe (nor am I alone) that that license is compatible with the CC‑BY‑4.0 license in either direction, inbound or outbound. It woud therefore be better to license all calibration data and related material under CC‑BY‑4.0 to enable its widest possible re‑use. Please note this point @Chris.Gordon-Smith. The doctrines and practicalities of open science should be uppermost.

The consultation documentation (the first URL above) suggests that “industry” is envisaged as the sole class of stakeholder. Given that most development in the openmod community takes place within academia, this seems to be an unnecessarily restrictive view.

The question of the standard patterns of usage arises. The sociology, if you like! Will there be some official set of standard occupant behaviors provided? And how will these various usage patterns be determined, validated, selected, and applied? This matter is particularly important because the methodology is to be also used for estimating carbon footprints and for tracking sector‑based progress towards net‑zero.

Similar remarks apply to the need for a nationwide inventory of building stock and associated characterizations. And again, under CC‑BY‑4.0 licensing for maximum re‑usability, assuming any privacy issues can be worked around in some technical way.

I guess the methodology will naturally update itself as the background climate data evolves (but not necessarily upwards) over time.

In the context of the noticeably increasing practice of conveying official regulations through reference software, it is worth noting that the Australian National Electricity Market (NEM) specification is defined using a model that only runs on the proprietary PLEXOS software. So the move here to open source in these circumstance is clearly laudable.

To underscore, it is paramount that the code, the data, including building stock and occupant inventories, and the policy and occupant behavior specifications are all under genuine open and interoperable licensing!

This openmod lightning talk (00:05:33) might provide some useful background: https://forum.openmod.org/t/vienna-2023-workshop-four-minute-lightning-talks/3622/20 And also the “built‑environment” tag on the forum too: https://forum.openmod.org/tag/built-environment

As a digression, I think that official agencies should abandon Twitter/X as a communication channel for public outreach. Twitter/X have not signed the European Commission’s voluntary code of conduct and commentators expect the Commission to take substantive action on legislative non‑compliance in due course (although admittedly under EU law).

Finally, others in the openmod might like to read the executive summary from the consultation document. The software design strategy, envisaged use‑cases, and rolling development approach (until somehow periodically frozen in legislation) are inspired. That said, there are quite a number of operational issues to resolve, particularly in relation to any mandated use in building performance standards.

HTH, R

Thanks for these extensive comments Robbie. There is plenty of food for thought there!

Overall, the idea of an officially maintained simulation environment to be used for regulatory purposes and developed on an open source basis seems very positive. I’m not sceptical about that.

But I am sceptical about replacing a 200+ page model specification document with Python code. Doing so would have (at least) the following drawbacks:

  • The code (and therefore the legal reference document) would only be understandable to Python programmers
  • If there was a bug in the code it would have a legal standing, even though it may be obvious that what the code does is inappropriate. (The comment about the Horizon project seems relevant in this context)

These drawbacks seem very serious to me, and so I wonder whether this is really what is intended. But the text you have quoted from the consultation seems clear enough.

By the way, I do think there may be scope for automating documentation to some extent. For example, in the C++ (and Python) worlds tools such as Doxygen can automatically generate documentation from program code. However, that’s a far cry from saying the code is the ultimate legal reference

I would be interested in any other thoughts in the forum on all this, especially on making the code the ultimate legal reference.

Regards
Chris

1 Like

Comprehensibility, explainability, usability, and community

It goes without saying that comprehensibility, explainability, and usability are useful and necessary conditions for most public software — and especially so for software destined to be embedded in legislation.

And good software support is clearly the key to meeting these three objectives.

With the code remaining the ultimate reference, bugs included — as must be, in my view. Noting that statutes, annexes, endorsed standards, and legal judgments are not free of errors either.

In general terms, some software support is socially remote — communicated through coding style guides, technical documentation, usage notes, static examples, and worked examples contained in interactive media such as Jupyter notebooks.

But most software support these days would class as socially active — in which a project community forms around the codebase and undertakes a range of often one‑on‑one assistance to specific requests. And most projects thus offer a number of forums for developers and users to engage, including mailing lists, chat servers, discussion forums, and public code repos with issues trackers.

That same project community normally tackles housekeeping tasks too. And will often tidy the codebase, develop and maintain naming conventions, and service the various forms of documentation and worked examples mentioned earlier. And also devise test suites, hunt for bugs, and respond to issues submitted. And so on.

Package management and related logistics are also handled by the project community. And a project will also generally need to decide whether a GUI, web‑deployment, or mobile device support is feasible and something to commit resources too (a debate that is always ever‑present in the openmod community).

The sum of these efforts necessarily magnifies and broadens as the audience moves beyond modeling geeks. The Home Energy Model (HEM) is at the other end of the spectrum and outreach will likely need to embrace architects, designers, building services engineers, legislators, policy analysts, law practitioners, various professional bodies, academics, environmentalists, the interested public, and many others.

So that central to the success of the HEM project, in my view, will be the development and nurturing of a supportive community. If such a community fails to materialize, the project will remain broadly opaque, untidy, unresponsive, and doubtless subject to complaints of intransparency and worse.

That community will also need to commit more resources than usual to producing formal and complete documentation that services a wider range of needs, use‑cases, and technical backgrounds than would normally be the case.

I suspect the greatest challenge for those sponsoring this project is to begin from the outset with a genuine open source ethos. And that means letting go entirely of the logistics at least and trusting your contributing devs and users completely. Interesting times, as they say!

And finally, this is not mission-critical software, unlike say the Linux kernel and any number of others open source projects that are used to control devices like cars and satellites.

Thanks for these useful comments.

One point that may be worth discussing further is the question of the ultimate legal reference. The code (including bugs) must of necessity be the ultimate reference for what the code actually does. But what is the ultimate legal reference for what it should to do?

Some brief personal notes may help to set my notes in context. I worked for many years on IT systems in container shipping. This left me with the view that the formal project documentation is the ultimate legal reference. In cases where there were system problems, the formal project documentation was always taken as the ultimate statement of what the system should do. If it had come to a court case (it never did) it would have been the documentation that would have been taken to define what the code should do. If it had turned out that the formal project documentation was wrong, that would have been recognised and the documentation and code changed as appropriate.

However, the HEM project is considering making the code the ultimate legal reference. Coming from the background I’ve mentioned, that seems odd to me. But I don’t want to jump to conclusions.

The case against making the code the ultimate legal reference seems obvious (e.g. only Python programmers can understand it). But what is the case for?

Here I think there may be another distinction to be made depending on the context in which an ultimate legal reference is needed. Any output from the model used for regulatory purposes will of necessity reflect what the model actually does. If, for example, a building plan receives an incorrect certification due to a bug in the code, it is common sense that a builder who goes ahead with the plan is not at fault. So in that context it seems reasonable to take the output from the code (e.g. a building certification document) as the ultimate legal reference in case of any dispute over whether the development should have gone ahead.

This line of reasoning leads me to the following provisional view:

  • The formal and complete documentation that Robbie mentions is necessary and should be the ultimate legal reference for what the model should do.
  • The ultimate legal reference in terms what happens beyond the modelling world (eg in the case of the builder mentioned above) may in some cases be the output actually produced by the model (eg for certification purposes).

Any further thoughts on this would be very welcome.

Which modeling artifact should take legal priority?

As far as I am aware, writing an open source software project into legislation is new territory in terms of jurisprudence.

First note that statutes and regulations are different. In the UK, regulations are used to give tangible effect to the various policies laid out in statutes. So I am assuming that the Home Energy Model (HEM) will be introduced via regulation. Even if that supposition is wrong and the HEM is specifically referred to in statute, most of what is said below would still apply.

An example of a policy objective covered in statute would be some minimum level of thermal performance for a new dwelling relative to some defined occupancy profile and physical location.

There are perhaps six artifacts in play in terms of HEM — all evolving and all subject to periodic versioned releases:

  • the technical specification for the program
  • the codebase plus core datasets
  • the developer documentation
  • the user documentation
  • the python interpreter and programming libraries deployed
  • the system environment

I have intentionally combined the python code and core data into one entity because they are functionally inseparable. A wrong data item is as potentially damaging as a faulty line of code.

The first question is whether one particular artifact needs to be accorded priority under law. Indeed, why not nominate the entire set of artifacts from one combined release. I suspect there is considerable merit in treating the entire project as one for statutory purposes. That way, if the specification or code or documentation or python interpreter is defective for some reason, then the aggregate can be used to determine the intended purpose.

The software bill of materials (SBOM) approach used in cybersecurity legislation adopts a similar line: that the entire software stack is in scope.

A common programming mantra is that the code is king and all else is secondary. There is no real reason why this particular truism should apply in this case.

If one artifact is to be singled out, which should it be? The key contenders are the program specification and the codebase plus dataset artifact, duly versioned together. I lean towards the implementation, rather than the specification, because that is what produces the answers.

The question of bugs in whichever artifact is both material and irrelevant. Material in the sense that serious bugs can corrupt both applications and results. And irrelevant in the sense that finding and fixing bugs is part of life — and better to have that process out in public for full scrutiny than hidden behind proprietary software and consultant reports in the case of specific analysis.

It is worth commenting that legal interpretation is sometimes highly vexed. And there are many judicial rulings that have taken legal commentators by suprise. So primacy does not necessarily equal certainty.

In summary, I lean to the idea the entire set of artifacts be given equal status under law. While I acknowledge that that is not a common proposition in law where usually one language or translation or text or artifact is accorded primacy. But I want to return to the question of why one artifact needs to be singled out as the arbiter of correctness. This is not a multilingual contract.

On a technical note, persistent software hash identifiers (SWHIDs) should be used to specify artifacts and parts of artifacts.