Open development environments to preserve global energy supply and vital natural resources

Hey Open Energy Modelling Initiative,

we are a group of former employees of StreetScooter and other engineers from the Aachen / Cologne area and have set ourselves the goal to create a list of all open technologies in the field of sustainable technologies. You are one of the few open communities in this field. Therefore we are looking for cooperation with you to complete the list. A large part of the list also contains the area of energy modeling. We would be very interested in your feedback. Do you know any other projects or categories that are missing here?

You also find more information on our blog post:

2 Likes

Hi Protontypes people! I very much like your idea of the sustainable innovation using FOSS practices. Thank you for sharing the awesome-sustainable-technology project! It looks really interesting.

I have an impression that the list could benefit from inclusion of the ā€œacademicā€ climate sources. They may be not exactly awesome in the FOSS sense, but they form the base of our knowledge about the climate and are in many cases open and free for researchers.

I would suggest the following addition to the existing Further Sustainability/Open Resources.

Station records

That is the longest and often the most reliable climate data source. Many national meteorological services have given a free assess for researchers to their data thanks to the World Data Center (WDC) program. It may be tricky to extract the data as the interface may be available on the national languages only like for Poland or Russia. Fortunately, there are some synthetic datasets. To the best of my knowledge, the GHCN project gives an assess to the most comprehensive global-scale archives.

Gridded data sets

That is a certain compromise between the directness of the observations and the need for spatially-distributed climate information. One of the leading global dataset of this kind is the archive of the Climatic Research Unit (University of East Anglia) and NCAS

The most detailed gridded observations data archive for the whole Europe is E-OBS.

Reanalysis projects

Reanalysis restore fields of the meteorological parameters from the first principles solving the system of equations for the mass/momentum/energy transfer in the atmosphere with the observation data being used as a special kind of boundary conditions. That mean a possibility to obtain a picture for the parameters which were/are not monitored with the ā€œclassicā€ meteorological station, e.g. hourly wind speed data.

An assess to the leading reanalysis products is provided by the European Center for Medium-Range Weather Forecasts (ECMWF) and NOAA Physical Sciences Laboratory.

Climate models archives

One of the most impressive open data initiative in the climate science is the Coupled Model Intercomparison Project (CMIP) which has allowed an access to the global climate outputs for the whole research community. The global climate models give a base to (almost) all we may know for the climate of the future. Now the CMIP project is in the phase 6.

Iā€™m not quite sure whether inclusion of these data sources is consistent with the project policy. If it is Iā€™ll be glad to make a pull request.

thanks @ekatef .You would really be closing an important gap with your contribution. In our search, we have indeed focused too little on datasets in this area. This is also because our background is more in sustainable mobility and renewable energy. I would be very pleased if you could contribute your knowledge here. I even think that this should be a category of its own. I am not sure about the name yet. I think in the future we would like to subdivide the datasets into different categories. What do you think of the category ā€˜Datasets of Ecosystemsā€™ or ā€˜Earth Observation Datasetsā€™?

Iā€™m not quite sure whether inclusion of these data sources is consistent with the project policy.

I have looked at everything you have posted. The posts fit the list really well.
From my point of view a dataset is ā€œopen and freeā€ if I am able to download and process them as a private person. Ideally you also allow free use of commercial use but this is not absolutely necessary. I do not want to be as strict with the data sets as I am with the software projects.

I would be very happy about a pull request from you.

As the meaning of the data sets becomes more and more clear to me, I have replaced the term ā€œTechnologyā€ with ā€œProjectsā€. Also through the feedback from others in this community like @robbie.morrison it became clear to me that not only the technology but also the datasets play a central role. I will also search more intensively for datasets for the other areas on the list.

1 Like

As @ly0n indicates, we had some offline traffic. In which case, I would like to highlight three themes, namely that:

  • there is no bright line separating data and models
  • data curation is as much a community activity as building system models is
  • powerful data management and interoperability tools are currently being developed, both generic and domainā€‘specific

On the first point, the climate data of most use to system modelers has all been processed by general circulation models. Raw station records, in contrast, are rarely of much interest. (Indeed I view the climate data versus models debate in society at large as somewhat misleading.)

2 Likes

@ly0n, many thanks for such a detailed response. I have added a ā€˜Datasets of Ecosystemsā€™ section as a pull request with a slightly revised climate datasets.

@robbie.morrison, thank you for sharing the data-model discussion. I think, the data/models distinguishing issue itself is rather of theoretical than practical importance. What is important, however, is that the GCM outputs are not intended to be used as forecast tool directly due to some intrinsic limitations, such as coarse spatial resolution. That is well known to the climate researcher but not quite obvious to the energy modellers (including myself :slight_smile:).

Some quite sophisticated techniques (bias correction, downscaling, ensemble approaches etc.) should to be applied towards the GCM datasets to justify practical calculations. So, it seems to be a good idea to add some open source post-processing tools like climate4R to the list, as well .

Another, a more advanced approach towards implementation the climate data into the energy modelling workflow is to use the climate datasets processed to cover one of the energy modelling requirements. @matteodefelice has recently published a comprehensive climate-driven-energy-datasets project which collects a wealth of the freely available energy variables datasets.

1 Like

thanks @robbie.morrison and @ekatef for your help.
The list is a great success going viral on reddit, hackernews and multiple other websites worldwide.
Just google for ā€œOpen Sustainableā€ or ā€œOpen Sustainable Technologyā€ and you know what I mean.
The domain is now https://opensustain.tech/.
It would be nice to continue working with you on further improvements. Our plan is to improve the website and also to add further content on the website like posting new project, open access paper in this domain and so onā€¦

1 Like

@ly0n Useful work! As far as the energy system models go, there are no bright lines between modeling, analysis, optimization, and monitoring and control. I guess down track you might consider databasing the information and making it filterable and searchable to provide a more nuanced user interface. Quite a number of the projects listed also have entries on wikipedia and linking through might be useful tooĀ ā€” if only because the information on wikipedia tends to be maintained at some level.

2 Likes

Providing a better user interface is on the roadmap as making the data accessible via a proper database also. It would be nice to have such a user experience:

Iā€™m also in contact with https://www.force11.org/group/open-climate-knowledge to ā€œcompleteā€ the list. We have many problems to find the projects that are not hosted on gitlab.com or github.com. Many institutions use public but self-hosted git instances. This makes it much harder to find them. Thatā€™s why we are gone search the open access papers by key terms from the list and ā€œgitā€ ā€œsvnā€ or ā€œopensourceā€.

2 Likes