Your opinion on current open models!

Modelling
1

Dear openmod-community,

Historically, our institute (IEK-STE at Forschungszentrum Jülich) has been developing and using mostly closed modelling approaches (e.g. THE IKARUS PROJECT [1] and closed datasets for scenario-based modelling of the energy and power sectors.
Partly inspired by the results of @stefan.pfenninger 's ‘write-a-thon’ during the openmod workshop in Frankfurt and the resulting publication Opening the black box of energy modelling: Strategies and lessons learned [2] (also co-authored by our former group leader @Heidi_Heinrichs) and some younger researchers’ activities in this field, an internal process of vivid discussions has been started as to whether or not our institute will become more active in the field of open modelling and open data.

We do not intend to start modelling from scratch once again (since this is what most colleagues did in the past and their models became unused after they left), but rather join a community around an existing approach and contribute to its development and also share our data. Therefore, we started by evaluating the paper-given list of currently available open modelling approaches regarding their suitability of our modelling questions and needs. This list was then narrowed down to the following three model frameworks

  • OSeMOSYS,
  • oemof, and
  • Calliope.

As a next step, we are hoping that you might help us with some constructive opinions:

Which of the above-mentioned frameworks seems to be most appropriate and flexible to model Europe’s energy system from bottom-up?
That is, regarding demands from private households, the industry, all forms of land and air traffic, but also regarding the transformation and distribution sectors such as the refinery sector, and both heat and electricity systems?

All views and opinions on this topic (may them be controversial or not) are highly welcome and appreciated!

Thanks in advance, also in the name of my numerous colleagues!

Fabian

1 Like
2

First I have to say, that I am an oemof developer and therefore highly biased :wink:

So from my subjective point of view oemof is the framework of your choice, even though all of the frameworks you mentioned will be suitable to model such an energy system.

Even if you don’t want to start from scratch at some point, you might need additional functionality. We highly welcome contribuitions and tried to build oemof in a way that you can easily add new functionality to your model:

  1. Oemof has a very flexible basic structure which does not bind to sector. You can easily build up gas, electricity, heating and other system and connect them with different kind of components.

  2. If you miss functionality you can just add it to the framework in a simple way.

  3. Oemof is community based, which gives you the opportunity to participate in the development process.

For further questions you can virtually meet us at github and in our chat or you can come to our next meeting in Magdeburg. It is open for anybody, who is interested.

European Model (AT, BE, CH, CZ, DE, DK, FR, LU, NL, NO, PL, SE):

3 Likes
3

PyPSA didn’t make the initial cut, but I’d just like to point out that PyPSA can also do other energy sectors:

Synergies of sector coupling and transmission extension in a cost-optimised, highly renewable European energy system

We’ll be extending it further into non-electric industrial sectors very soon.

One problem with PyPSA and also with oemof and calliope (to my knowledge) is that you cannot optimize over multiple time horizons, like you can in IKARUS and TIMES.

At the moment only OSeMOSYS does this, to my knowledge.

We will add this functionality within a year to PyPSA - it’s just a case of tracking asset lifetimes and build years.

2 Likes
4

Hello Fabian, all. What a can of worms! But, as I have no substantive direct association with any open modeling projects, let me add my thoughts. Some criteria to begin with:

  • nature of the model : mathematical programming (typically MathProg) versus object-oriented (typically python)
  • electricity network representation : transportation or load flow
  • track record on modeling Europe
  • community status : because that is central

But, before I begin in earnest, I would suggest adding PyPSA to your shortlist (this comment drafted before Tom Brown’s posting above).

Nature of model   I personally make a wide distinction between mathematical programming (MP) models and object-oriented (OO) models. Only OSeMOSYS fits the former camp. The benefits of MP models are a short and comprehensible codebase (as little as 400 lines) and being ready extensible to other optimization paradigms (such as stochastic programming). The advantages of OO models are self-contained classes, design benefits arising from same, and much more sophisticated technology (and other entity) characterizations (like fuel cycle dependent ramp rates for legacy nuclear). Also enhanced opportunities for confirming model integrity, screening input data, and monitoring the runtime behavior of entities. Conversely, the codebase is large and poor software engineering and programming practices can create severe maintainability issues. OO models have better access to non-classical solution techniques, including rule-based dispatch and genetic algorithms. MP models are necessarily classical in that sense.

Electricity network representation   Treating electricity flow as transportation (DC) or load flow (AC) may be a key criteria. The python models on your shortlist support AC load flow but individual model instances might revert to DC load flow for reasons of efficiency. And while MP models can easily implement AC load flow (as evidenced by nodal pricing algorithms), I don’t know where OSeMOSYS stands in this regard.

Track record of modeling Europe   This is not a comprehensive review, but here are some pointers to the (rather thin) literature and other statements:

  • OSeMOSYS: An EU-28 model covering western and central Europe is in planning, funded as part of Horizon 2020 and falling under work package WP7 of the REEEM project
  • PyPSA: Brown et al (2018)
  • not shortlisted but nonetheless building European models: DESSTinEE, Dispa-SET, EMMA, GENESYS2, URBS

Community status   The various projects under consideration are at various points along the community building spectrum. OSeMOSYS is the most advanced in this regard, but with much of its community resident, conceptually at least, outside Europe. OSeMOSYS was also the first project to confront community governance, having appointed a community development manager (Agnese), established a steering committee, and embarked on a formal process of prioritizing development, endorsing specifications, and reviewing contributions. But perhaps not as free wheeling as the other projects in this regard? That said all the projects you mentioned are committed to building developer and user communities.

Finally to comment on model coupling, a topic that is attracting increasing attention. My bias (and running against the current tide) leans toward single codebase models for two reasons: conceptual and technical. A single codebase model (MP or OO) is, by definition, fully integrated. But tight runtime coupling using two or more models is likely to be extremely expensive in terms of process management and interprocess communication (IPC), leading to slow performance.

HTH, Robbie

Some references

Brown, Tom, David Schlachtberger, Alexander Kies, Stefan Schramm, and Martin Greiner (16 January 2018). “Synergies of sector coupling and transmission extension in a cost-optimised, highly renewable European energy system”. Preprint.

1 Like
5

Hi all,
as Robbie states the community status as central I would add some criteria that we think are necessary to evaluate it:

  • is the organisational structure of the community made clear?
  • How can I exchange with the others?
  • How are decisions made?
  • how can I contribute?
    Explanations can mostly be found on the homepage and a lot of insight can be gained when you go to the common modelling site (GitHub or similar) - you see if questions and pull requests are open for the community and how much is going on there.
    best
    Berit
1 Like
6

This thread is interesting because it suggests to me that perhaps the information available online (e.g. in publications, model documentation, the Openmod wiki, etc) is insufficient to really draw out the differences between the available modelling tools in a way that helps informed users to make a choice.

I certainly agree that PyPSA should be on the list!

To more clearly place Calliope within these other options, I think we try to balance appealing to both Python-literate modellers (documented API, well-tested code base with >90% coverage, etc), while also appealing to modellers with no interest in or knowledge of Python, by allowing specification of a model in human-readable YAML files, being able to dump results to CSV files and generate interactive Plotly plots after solving a model (see docs from our soon-to-be-released version 0.6.0 on analysing results – Plotly is not yet in the current stable release).

As in the case of PyPSA, multiple time horizon planning could be added if needed.

Some other functionality currently in development (also see our GitHub issues and project boards):

  • Robust optimization, allowing us to account for uncertain parameters along different scenario pathways.
  • Piecewise linearization, to better represent complex technology characteristic curves (such as the nonlinear efficiency curve of some technologies).
  • Easily implementable sensitivity analyses, including the ability to package up many models into a single NetCDF file to easily deploy onto a high-performance computing cluster.
  • Coupling to Renewables.ninja, to feed in renewables resource data for a model based on node locations.

So, I’d say one aspect to consider is the trade-off between what questions you think you’ll be mostly looking at and the degree to which you want to your model users writing Python code.

2 Likes
7

Dear Fabian, all,

I am happy to read through this thread and all the interesting comments and relevant points of discussion that you already brought up.

From my side, I would like to add few information regarding OSeMOSYS, hoping they can be useful for you to take a decision.

The OSeMOSYS model for Europe (OSeMBE) should be released by the end of April and for now it has been developed with a top-down approach. It is of course quite flexible (as typical of OSeMOSYS models) and could be used as basis for more detailed models.

Concerning the Community around OSeMOSYS, a preliminary structure is already available. Some improvements in e.g. the accessibility of the GitHub repository and Documentation (manuals, Q&A sections) are currently under implementation. This should facilitate the interactions in between different types of users (developers, modelers, and policy makers) and better allow for open discussions on decision regarding the future development of the tool and the community itself.

To be highlighted then, is the role that OSeMOSYS has played in supporting policy makers and providing capacity building programs in several countries around the world. Therefore, the target audience is a bit peculiar, compared to other energy modelling tools, and it justifies the need for OSeMOSYS to keep being a very flexible tool, easy to understand for non-experts so that modelling results can be fully trusted. Moreover, it will be increasingly important for the Community to work on further developing an extensive range of teaching material and documentation, video tutorials etc. to ensure a fast learning curve.

Another positive thing of OSeMOSYS is the fact that it is currently available in three different programming languages (.e. MathProg, Python, GAMS) giving you a bit of flexibility also in choosing which one to use and enhance for your work, depending on the programming language you are more familiar with.

Finally, the following paper: From the development of an open-source energy modelling tool to its application and the creation of communities of practice: The example of OSeMOSYS has just been published, providing you with information on latest developments, applications, community structure and shorter-term modelling (looking at changing the long-term modelling concept of timeslices).

Best regards,
Agnese

2 Likes
8

Hello Agnese. Seeing I have that citation in markdown, here it is. Robbie

Gardumi, Francesco, Abhishek Shivakumar, Robbie Morrison, Constantinos Taliotis, Oliver Broad, Agnese Beltramo, Vignesh Sridharan, Mark Howells, Jonas Hörsch, Taco Niet, Youssef Almulla, Eunice Ramos, Thorsten Burandt, Gabriela Peña Balderrama, Gustavo Nikolaus Pinto de Moura, Eduardo Zepeda, and Thomas Alfstad (April 2018). “From the development of an open-source energy modelling tool to its application and the creation of communities of practice: the example of OSeMOSYS”. Energy Strategy Reviews. 20: 209–228. ISSN 2211-467X. doi:10.1016/j.esr.2018.03.005.

1 Like
9

Dear all,

a sincere thank you for your well thought-out and extensive feedback!
That is pretty good food for thought and will be fed into our internal discussion.
I’ll come back to you as soon as we’ve worked through the details and (hopefully) came to a conclusion…

Thank you again, also from my colleagues!
Fabian

10

An update for OSeMOSYS. The OSeMBU reference model covering western and central Europe was announced on 27 April 2018 (Beltramo 2018, OSeMOSYS 2018). The model uses the MathProg implementation of OSeMOSYS but requires a small patch first. The model refers to the current situation with the United Kingdom still part of the European Union. Robbie.

References

Beltramo, Agnese (27 April 2018). “first OSeMBE EU-28 model released”. openmod-initiative@googlegroups.com (Mailing list).

OSeMOSYS (2018). The open source energy model base for the European Union (OSeMBE). OSeMOSYS. Stockholm, Sweden.