Hi all,
I built a six region, hourly dispatch model considering one year as time horizon.
I used the simple_dispatch example as a basis, and then I modified it to fit my needs.
The model runs correctly just once using the same console, if I run it twice I get the Memory Error.
I have no errors if I run it more times but reducing the time horizon.
Is there a way to free memory space after collecting the result?
(I am using a 32 bit version of Anaconda)
Thanks!
Valeria
Welcome to the oemof forum 
Normally it is possible to run multiple models in a row and in my case the memory is automatically emptied after each run. You just have to make sure that you dump your results instead of keeping them in your memory but I guess you already do this.
There are memory profilers available in python. Here is a blog where topic 6 might be interesting for you.
There are also memory trackers for your OS (Windows, Linux, Mac,…) or just a small application that shows the memory usage so that you can follow it during a python run and check if the memory use is higher at the end (when you start your second run). I guess you are close to your limit.
The memory usage depends on the number of variables and constraints, so a storage might use more memory than a transformer. So counting the regions does not really help.
Let’s proceed here and now via e-mail. I’ll reply to the latest mail from Matteo:
Hi Cord,
I’ve installed pyomo version 5.5. but I get always the same memory error (this is actually the error message that Valeria, a master student that works with me, got this morning. We checked and she has already installed the version 5.5 of pyomo):
File “”, line 1, in
runfile(‘C:/Users/VCasalicchio/Desktop/GA+oemof_NoParall_20_11_18/GA+oemof_NoParall/FunctionForGA.py’, wdir=‘C:/Users/VCasalicchio/Desktop/GA+oemof_NoParall_20_11_18/GA+oemof_NoParall’)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 705, in runfile
execfile(filename, namespace)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 102, in execfile
exec(compile(f.read(), filename, ‘exec’), namespace)File “C:/Users/VCasalicchio/Desktop/GA+oemof_NoParall_20_11_18/GA+oemof_NoParall/FunctionForGA.py”, line 215, in
pop, ff, hist = GA(X, Simulation, objectives, Npop, Ngen)File “C:\Users\VCasalicchio\Desktop\GA+oemof_NoParall_20_11_18\GA+oemof_NoParall\GA.py”, line 102, in GA
fitnesses = list(map(toolbox.evaluate, pop))File “C:/Users/VCasalicchio/Desktop/GA+oemof_NoParall_20_11_18/GA+oemof_NoParall/FunctionForGA.py”, line 164, in Simulation
om.solve(solver=‘cbc’)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\oemof\solph\models.py”, line 184, in solve
results = opt.solve(self, **solve_kwargs)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\opt\base\solvers.py”, line 599, in solve
self._presolve(*args, **kwds)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\opt\solver\shellcmd.py”, line 196, in _presolve
OptSolver._presolve(self, *args, **kwds)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\opt\base\solvers.py”, line 699, in _presolve
**kwds)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\opt\base\solvers.py”, line 770, in _convert_problem
**kwds)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\opt\base\convert.py”, line 110, in convert_problem
problem_files, symbol_map = converter.apply(*tmp, **tmpkw)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\solvers\plugins\converter\model.py”, line 96, in apply
io_options=io_options)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\core\base\block.py”, line 1681, in write
io_options)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\repn\plugins\cpxlp.py”, line 176, in call
include_all_variable_bounds=include_all_variable_bounds)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\repn\plugins\cpxlp.py”, line 722, in _print_model_LP
con_symbol = create_symbol_func(symbol_map, constraint_data, labeler)File “C:\Users\VCasalicchio\Anaconda3\lib\site-packages\pyomo\core\kernel\symbol_map.py”, line 82, in createSymbol
self.bySymbol[symb] = weakref_ref(obj)MemoryError
Matteo
Cheers
Cord
You said that your “breaking” model is embedded within a generic algorithm.
1.) Did you check if one of the simple examples is running with CBC after updating to the latest version? See: https://github.com/oemof/oemof-examples
2.) Could you also check another solver e.g. GLPK?
If a simple example is running through with both solvers the problem can be attributed to some combination of pyomo and your algorithm e.g. due to memory usage in parallelisation.
Cheers
Cord
Hi Cord,
Yes, the model we have created is based on Excel reader example and works perfectly with CBC solver
GLPK and Gurobi gave the same problem.
The issue has been solved installing Anaconda 64 bit that allows to use more memory as described here: How to solve the memory error in Python - Stack Overflow
Thanks a lot for your help and quick replies,
Matteo