Network of interoperable and distributed database infrastructure for energy research

Workshop
, , , , ,
1
  • genre: break‑out‑group
  • title: Network of interoperable and distributed database infrastructure for energy research
  • presenter: @ludwig.huelk @danielhuppmann
  • background:

This breakout-group is dedicated to the RDM-infrastructure and databases. (RDM is research data management.) Focus is the collaborative use of the OpenEnergyPlatform for Open Research Data (ORD) and the interoperability of inhouse database systems like Ffe Open Data Portal and IIASA Databases.

An introduction of the Energy Databus shows a technical solution for distributed database infrastructure creating a metadata catalogue and enabling semantic searches and linked open data (LOD).

After discussing future community activities there is space to register existing datasets and experimenting with the API and database queries.

3 Likes
2

Thanks for proposing this session! I suggest that you co-chair this session together with the Data Steward of the #IIASA ECE program @phackstock.

Also, at the #berlin-2020 workshop, we started a prototype to connect the Open Energy Platform and the pyam package (https://pyam-iamc.readthedocs.io) - I’d be very curious if we can revive that effort in #vienna-2023!

1 Like
3

I am not at the workshop but would like to mention, that we developed a set of crawlers of typical open-data which can be set up as an inhouse-database used for energy system modelling:

It has crawlers for Weather (ECMWF, renewables.ninja), EEX data (provided you have a subscription), frequency data, opsd, ENTSO-E and ENTSO-G and a few more.

The crawlers can be run on a weekly basis to always get the latest updates of the selected data sets.
It requires to have a PostgreSQL based TimescaleDB instance running somewhere in your institute (accessible for all researchers and modelers).

Might be interesting for other people too, and we would like to cooperate with this, if someone is interested :slight_smile:

2 Likes
4

The result from the 2020 openmod break‑out group can be accessed in this repo:

In this session we will focus on the different approaches of data structures (OEP → flexible database / IAMC → fixed model) and evaluate where and how we can combine them.
There were a lot of openmod sessions in the past we will have in mind:

Existing approaches:

5

At the workshop we discussed the issue of data models.
Here is the compiled list of requirements that should be considered:

  • Regional dimension
  • Time dimension
  • Methods (sum, avg, min, max)
  • Ranges
  • Model types (optimisation, agents)
  • Scenarios
  • Provenience
  • Copyright → Attribution
  • Good Scientific Practice (GSP) → Citation
  • Project/Consortia