Model usability

The are now several attempts within the openmod community to improve the usability of various energy modeling frameworks. This topic collects a few of those strands and presents them by project team in alphabetical order.

The motivation for this posting ranges from simple cataloging and an attempt to share experiences — through to helping examine opportunities to improve cross‑project consistency where this might be useful and productive and perhaps even develop and share workflow practices and supportive tooling across projects.

Some material presented here covers user interface design and local workflows. But some also looks beyond to consider portals, web‑services, and community processes. In other words, the UX in total. That said, the focus here remains firmly on users and particularly those users relatively new to energy system modeling.

Usability, in this context, spans the entire range of user interfaces from command‑line, spreadsheet front‑ends, and through to dedicated GUI environments, both local and server‑based. There is no presumption that one paradigm is necessarily superior. As can be seen, different projects have adopted different strategies.

Allied to these initiatives are efforts to promote energy system analysis in Africa and the global south more generally. Indeed, user bases are beginning to broaden out well beyond the original development teams.

Energy system analysis is now also starting to be taught within undergraduate courses and as upskilling modules for policy analysts.

In terms of language, “framework” refers to the codebase, “model” indicates a framework that has been duly characterized for some system of interest, and “scenario” refers to variations on the assumptions that underpin a specific model. And the usual analytical practice is to run and compare a number of scenarios. That same vista also reflects a general shift from developers to users.

I should confess I come from a background of UNIX, SSH, command‑line compilers, derelict codebase management, and rudimentary revision control. So this is a journey for me too. And my apologies in advance if I have omitted or misrepresented any of the projects or their objectives — indeed, please correct as required. And feel free to upload your screenshots too!

2 Likes

This is a wiki‑post that any registered openmodder may edit. Moreover, the current post is just meant to be a skeleton really.

Projects possibly missing include Spine?

Calliope team

The Calliope project has developed a command‑line workflow that utilizes Python scripts and relatively intuitive YAML files for data interchange. Some details can be found in this 2021 video:

NREL has partnered with the Calliope team to develop an open source web‑interface to the Calliope framework. This work is part of the NREL engage project which seeks to improve stakeholder engagement in relation to energy planning and the kinds of community trade‑offs that can arise. Engage has been trialed in Hawaii. More background on the engage website:

  • NREL (ongoing). Engage. National Renewable Energy Laboratory (NREL). Golden, Colorado, USA.

A screenshot of the engage web‑interface to Calliope follows:

placeholder-image.02

GenX/PowerGenome team

This umbrella includes the GenX framework and the PowerGenome project to harvest and supply data. The analytical focus is the United States. As of early‑2022, PowerGenome only supports GenX but there are plans to broaden its interface to interact with other modeling frameworks.

oemof team

The oemof team are developing spreadsheet interfaces:

  • Spreadsheet Energy System Model Generator (SESMG) which embeds additional functionality
  • deflex spreadsheet‑based modeling toolbox

In addition, an oemof‑based open‑plan‑tool is being developed to facilitate interactions with stakeholders.

Open Energy Family

The Open Energy Family includes the Open Energy Platform (OEP) and the Open Energy Ontology (OEO). The OEP provides an API interface for programmatic interaction.

OSeMOSYS team

The OSeMOSYS umbrella includes the OSeMOSYS framework, the OnSSET framework, the U4RIA umbrella, and the emerging UNDP CLEWS community.

Efforts to develop a non‑command‑line interface are being pursued under the rubric of clicSAND and variants. The OSeMOSYS team are also developing starter kits for selected regions in Africa and elsewhere. See the publications below for details on both counts:

PyPSA team

The PyPSA umbrella includes the PyPSA framework, the linopy solver interface, and projects to promote wider uptake under the banners of PyPSA‑meets‑Africa and PyPSA‑meets‑World.

Users of PyPSA currently tend to be limited to those who are sufficiently fluent in the python language. There are plans for a web‑interfaced version. Some references follow:

TEMOA/Open Energy Outlook team

This is another umbrella with a United States focus. The TEMOA framework and associated Open Energy Outlook projects are currently developing a web‑interface.

1 Like

Here is related policy‑oriented initiative that traverses usability in the broader context of model reproducibility (by other teams) and interoperability (in relation to data and data standards):

1 Like

Within the open_MODEX project we have developed and tested a method for usability testing of ESM frameworks. Have a look if you are interested: Evaluating the Usability of Open Source Energy System Modelling Frameworks (RSER 2022)

3 Likes

Hi @sar_b Many thanks. The models covered in Berendes et al (2022) comprise:

The accompanying literature survey suggests that developers of open source energy system frameworks rarely mention the topic of usability in either formal or gray publications.

The authors deploy supervised structured testing to evaluate the usability of the five frameworks mentioned whereby users are assigned specific tasks and later asked to report their experiences based on using one (single framework approach) or several frameworks (cross‑evaluation approach). Participants can be further broken into developer‑users and non‑developer‑users, but this case study employs only developer‑users. All participants fill out the same comprehensive questionnaire.

The results are quite nuanced but “input data”, “conversion script”, and “error messages” ranked as major headaches. And “standardized input data” was often sought as a request.

For completeness, the full citation is:

Partly inspired by my experience building energy models, myself and a friend developed MOS, software that is designed to abstract away the “glue code” around optimization models more generally, placing the model behind an API and also providing a default user interface. Available to try out via Docker, and repositories are on GitHub MOS Software Documentation — MOS documentation

Wanted to share in case of interest to the community here. I am impressed by some of the interfaces mentioned above, was not previously aware of them.

1 Like