Hi open modellers,
Let me start by wishing you happy modelling in this new year!
After finishing my master thesis on explorative use of energy system optimization models, I’ve tried to push the use of ESOMs for policy advice at my current position (consulting engineer at Witteveen+Bos). However, the licensing costs for commercial application of Gurobi are very steep and are found restrictive. Unfortunately, we cannot bear those cost while also trying to expand our knowledge on the subject and develop the market.
I’ve been researching alternatives because of this. Of course, I am looking into the use of the very promising open source solver HiGHS. Preliminary results may indicate that performance for higher temporal resolution (hourly) and ~30 nodes is not yet acceptable for our use case. Whenever I’ve read about Gurobi and CPLEX, both of them are mentioned together in the category of “outstanding commercial solver performance”. I can’t seem to find any more specific information about the difficulty of utilizing the CPLEX-solver or its performance compared to Gurobi. Based on what I could find, I concluded that the costs of commercial licensing are significantly lower for CPLEX while it is considered in the same ballpark in terms of performance.
The lack of information on the application of CPLEX might be a logical consequence of the fact that it is very easy to obtain an academic Gurobi license and both ease of use and performance is great.
Does anyone have experience with using CPLEX as solver for PyPSA?
I will schedule a call with one of their sales reps to discuss their perspective on the matter and what solutions would best fit us but I wanted to get an unbiassed opinion on the matter too.
Thank you in advance!
Hi @Seth The following citations do not provide direct answers to your questions, but you might gain some clues from the given literature nonetheless. Regarding PyPSA and CPLEX, this third‑party solver is officially supported — as you are no doubt aware.
On various energy system modeling frameworks:
Macmillan, Madeline, Kelly Eurek, Wesley Cole, and Morgan D Bazilian (1 November 2021). “Solving a large energy system optimization model using an open-source solver”. Energy Strategy Reviews. 38: 100755. ISSN 2211-467X. doi:10.1016/j.esr.2021.100755. CC‑BY‑4.0 license.
This study explores the ability of the open-source COIN-OR linear programming (CLP) solver to compute solutions for the Regional Energy Deployment System (ReEDS) model—a large-scale, open-access electricity system planning model for the United States developed by the National Renewable Energy Laboratory (NREL).
PyPSA team (11 February 2022). Release notes — PyPSA 0.19.3 documentation. PyPSA repository.
release 0.19.3 states: “New features: Add open-source HiGHS solver.”
On solver interfaces:
Frangioni, Antonio and Sandrine Charousset (28 October 2021). plan4res SMS++: an open modelling library for evaluating long term electricity system costs and flexibilities. Italy: University of Pisa. EMP‑E 2021 presentation. Open access.
- SMS++ is a problem↔solver manager written in C++17 that “allows exploiting specialized solvers”, both third‑party and native
On optimization solver comparisons more generally:
Mittelmann, Hans (ongoing). Benchmarks for optimization software. Decision tree for optimization software. Website. Accessed 16 December 2020.
IBM (CPLEX) and FICO (Xpress) demanded that results for their solvers be removed and then we decided to temporarily remove those of Gurobi (Gurobi) as well. (emphasis added)
Mittelmann, Hans D (25 June 2019). Benchmarking optimization software: a (hi)story. Tempe, Arizona, USA: School of Mathematical and Statistical Science, Arizona State University. Presented at EURO 2019, Dublin, Ireland.
Thank you for that information, Robbie!
The EURO presentation of Mittelmann contained a lot of useful information. And what a drama, by the way. It’s a shame that even independent assessment of these commercial powerhouses is being prohibited.
Could you maybe elaborate on why you think the slides on SMS++ are relevant in this case? From what I can make up it is another modelling library that uses C++ but also uses external solvers like CPLEX.
Hi @Seth Regarding SMS++ … not really. But it looked interesting. I once wrote a energy system to solver interface in C++ after working with software that lacked that abstraction (cf Pyomo, JuMP, and the PyPSA-specific Linopy). But in the case of SMS++, there seemed to be additional features bundled? Adaptive too?
Sorry, I cannot look further. My bandwidth is fully taken up with software patents at present. And how this risk intersects with license choice.
The PyPSA crew have a discord chat instance as well. So if you need to drill down, that might be useful. That URL here on their wikipedia infobox:
And if you have your masters thesis as a PDF, I would be interested in having a look. My email: email@example.com
good luck, Robbie
Thanks again, Robbie. Good luck with the software patents!
I’ll join the PyPSA discord, good tip. I’ll send you my thesis
We went through a similar process as a small consulting firm. Ultimately we settled on FICO Xpress, which was by far the most affordable option compared to Gurobi and CPLEX, if you are looking for a single seat license. The performance using their barrier solver is more than adequate for relatively large PyPSA models.
Another option is to make use of IBM datapak, which includes a CPLEX license. Here you pay for compute hours in the cloud, however we found the process a bit more unwieldy than running models on a local machine.
I would be happy to pass on the details for our contact at FICO. (I have no affiliation to FICO).
Thank you for that very helpful response, Peter!
If you could send over the contact info of your guy at FICO, that’d be great!. I’ll send you a PM.
Good to know about the IBM datapak option, that might just be a flexible solution that we need.