Excel reader: dtypes of nonconvex flow parameters

Hello openmod community,

And a happy new year to everybody. I am currently working on an excel-reader to create an energy system. But it seems I am lacking understanding of how OEMOF handles nonconvex flows of a transformer component to build a MILP-problem model. Maybe someone can give me a hint how to fix my issue.

My Problem:
My script automatically reads in an excel file with components data. The script is based on the excel reader example from the example collection and is working without an error for LP-Problems.
But as I start to assign startup_costs to a transformer component in my excel file OEMOF fails to initialize the model later on. There is no error during creation of the components or while setting up the energy system. But once I try to initialize the model there is an error (see below). I feel like I am missing out on how to correctly add a nonconvex flow to my transformer component. Just to mention: the script throws no error when I assign a nonconvex flow to a source component. It seems to be specific to the transformer component.

My script & data:

transformers sheet of node_configuration.xlsx:

label active inputs outputs conversion_factors inputs.nominal_value inputs.max inputs.min inputs.fixed inputs.variable_costs inputs.startup_costs inputs.activity_costs outputs.max outputs.min outputs.fixed outputs.variable_costs
gas_boiler 1 b_gas b_heat 0,9 300 100 1000

relevant section of python script:

from oemof import solph
import os
from math import isnan
import pandas as pd

def main():
    xls = pd.ExcelFile("input/node_configuration.xlsx")
    nodes_data = {x: xls.parse(x) for x in xls.sheet_names}

    my_nodes = create_nodes(nd=nodes_data) #creates a list of all nodes

    run_model_optimization(nodes=my_nodes) #initializes and solves the model

def create_nodes(nd=None):
    nodes = []
    busd = {}

    #list of nonconvex-flow variables to filter parameters
    nonconv_flow_var = ["activity_costs", "startup_costs"]
    #list of linear flow parameters for filtering
    flow_var = ["nominal_value", "max", "min", "fixed", "variable_costs"]
    #list of solph node parameters
    node_var = ["conversion_factors"]

    #create bus components from 'buses' table
    for i, b in nd["buses"].iterrows():
        if b["active"]:
            bus = solph.Bus(label=b["label"])
            nodes.append(bus)
            busd[b["label"]] = bus

    #create transformer components from 'transformers' table
    for label in nd["transformers"][(nd["transformers"]["active"] == 1)]["label"].unique():
        t = nd["transformers"][(nd["transformers"]["label"] == label) & (nd["transformers"]["active"] == 1)]
        #set up dict for static node variables
        i_args = {}
        o_args = {}
        conversion_args = {}
        node_args = {"label": label} 

        for io in ["inputs", "outputs"]:
            for bus_label in t[io].unique():
                t2 = t[t[io] == bus_label].max(axis=0) #reduces df to one row with max value if there are multiple rows with same bus label

                # set static flow values
                flow_args = {x.split(".")[-1]: t2[x] for x in t2.index.values if x.split(".")[-1] in flow_var and (x.split(".")[0] == io) and not isnan(t2[x])}  # kwargs for flow & filter out all NaN values
                nonconvex_flow_args = {x.split(".")[-1]: t2[x] for x in t2.index.values if x.split(".")[-1] in nonconv_flow_var and (x.split(".")[0] == io) and not isnan(t2[x])}  # kwargs for nonconvex flow & filter out all NaN values


                # add nonconvex flow obj. to flow kwargs
                if len(nonconvex_flow_args) > 0:
                    flow_args["nonconvex"] = solph.NonConvex(
                        **nonconvex_flow_args)  # construct nonconvex obj in kwargs for flow
                if io == "inputs":
                    i_args[busd[bus_label]] = solph.Flow(**flow_args)
                if io == "outputs":
                    o_args[busd[bus_label]] = solph.Flow(**flow_args)
                    conversion_args[busd[bus_label]] = t2["conversion_factors"]
       
        # add flow obj to node kwargs
        node_args["inputs"] = i_args
        node_args["outputs"] = o_args
        node_args["conversion_factors"] = conversion_args

        # create component
        nodes.append(solph.Transformer(**node_args))
    return nodes

def run_model_optimization(nodes=None):
    solver = "cbc"  
    date_time_index = pd.date_range(start='1/1/2018', end='1/08/2018', freq="H")
    energysystem = solph.EnergySystem(timeindex=date_time_index)

    # adding nodes to the energy system
    energysystem.add(*nodes)
    model = solph.Model(energysystem) # <-- this is where the error occurs
    model.solve(solver=solver, solve_kwargs={"tee": True})
    ...


I think there must be something going wrong in the create_nodes() function. I designed it as such, that there is a nested collection and unpacking of relevant parameters in dictionaries with the **-operator. So at first a parameterized nonconvex flow is beeing put into the flow_args-dictionary before a flow object can be created. After that the relevant input/output flow is being used in a dictionary to create the transformer object with all additional component parameters in the node_args-dictionary.

Error message:

11:22:59-WARNING-DEPRECATED: Chained inequalities are deprecated. Use the inequality()
    function to express ranged inequality expressions.
    (called from /.../venv/lib/python3.9/site-packages/oemof/solph/blocks/non_convex_flow.py:330)
    ERROR: Rule failed when generating expression for constraint NonConvexFlow.min
        with index ("<oemof.solph.network.bus.Bus: 'b_gas'>",
        "<oemof.solph.network.transformer.Transformer: 'gas_boiler'>", 0):
        ValueError: Constraint 'NonConvexFlow.min[b_gas,gas_boiler,0]' does not
        have a proper value. Found 'True' Expecting a tuple or equation. Examples:
           sum(model.costs) == model.income (0, model.price[item], 50)

There seems to be a problem with the nonconvex flow but I don’t understand the error message. The debugger suggests the error occurs while constructing the BaseModel at the function call of _add_child_blocks().

Bildschirmfoto Debugger

Am I not aware of something in the routine of component creation? Is there something obviously wrong in my code?
However the documentation of the Flow class explains there is a significant change of the model if a NonConvexFlow object is added here. I wonder if this conflicts with my nested initialization of a MILP problem.

Any help is appreciated.

Best regards
Till

I do not have the time to check the whole code but it seems to me, that there is an error while mapping the parameters into the solph objects.

Debugging:

  1. Create a hard coded model with a minimum number of objects including your transformer with the nonconvex flow. Make sure everything works as expected. Just use a couple of time steps. If it works for 5 time steps it will work for 5000

  2. Now you can try to create the same energy system by mapping the data from your excel sheet. Check the **flow_args and make sure that they are equal to the parameters in the hard coded version. Double check the spelling of each parameter. Misspelled parameters will be ignored. If you initialise the objects exactly like in the hard coded version, everything should work fine because the model will not notice how you created the objects :smiley:

We are adding more error messages to the solph code so that it will be easier to find misspelled parameters or wrong input types in the future. Some of these changes will be part of the v0.5.0 release.

1 Like

Thanks for your suggestion Uwe! I finally found the cause of the error.

When pandas reads in the excel file it casts the columns with one or more NaN values as a numpy.int64 / numpy.float64 array. While mapping the values into my flow_args variable the dtype remained the same. Unfortunately solph can’t initialize the model with flow parameters of dtype np.int64.

My solution was to cast the variable as native float and thus the model can be initialized. e.g.:

flow_args = {x.split(".")[-1]: t2[x] for x in t2.index.values # code from above

flow_args = {x.split(".")[-1]: float(t2[x]) for x in t2.index.values # code with float cast

Maybe a dtype check or a specific error message in the model initialization would be helpful in future versions.

Thank you very much!

Could you provide a minimal example that causes the error message. This would help to write a check or to improve the error message.