We recently integrated the open-source HiGHS solver into PyPSA (see PR) and observed an incredible performance boost compared to alternatives. Promising is that the HiGHS solver team is actively improving the solver whereas most other open-source solver communities stopped developments.
Further performance check:
- 1year, 3h resolution, (~2920 snapshots), 3 nodes: CBC/CLP ~14min, HiGHS ~8min, Gurobi ~1min
- 1year, 1h resolution, (~8760 snapshots), 3 nodes: CBC/CLP ~130min, HiGHS 50min, Gurobi ~1min
To benefit the whole community, PyPSA colleagues are working on Linopy - a more generalised solver interface that is tailored for energy system models. This general interface will include HiGHS and other open and closed source solvers. One can say it’s a successor of Pyomo for these specific problem types as it accelerates the speed through harnessing the power of xarray, pandas, dask. Its official 0.1.0 release is coming very soon.
Open source solvers still can’t properly deal with long-timeseries problems. While we observed an extreme speed boost for short time series, this holds not true for longer time series. Gurobi solves in 1min an investment and dispatch problem with DCOPF that contains 5 nodes, 1 year, and hourly resolution, while HiGHS and others may need 2+ hours. This performance drop can occur because of dense-columns issues or other problems. Such bottlenecks can be removed by supporting the developer community behind the tools. For instance, the HiGHS team is currently searching for funding to further improve the IPM solver.
Thank you to @fabian.neumann , @lisa.zeyen , @fabianhofmann (PyPSA team) for supporting this development during the research visit at @tom_brown group. Thank you also to Julian Hall and Leona Gottwald (HiGHS team) for the interesting discussion. Finally, also thank you Daniel Rehfeldt (PIPS-IPM++) for the invite and discussion at the ZUSE Institute Berlin.
PyPSA meets Africa
University of Edinburgh