Running brightway2 models from other programs

Question

I want to run Brightway2 within an optimization framework (https://brightwaylca.org/).

Basically, I want to create a Python script that sends an input file to an external model (also in Python) and gets the outputs. The script will then write activity data into a Brightway database, and then run Brightway2 to get an LCA score. This score will then be used to update the input file based on an optimization algorithm.

Brightway2 seems uniquely qualified for this sort of project, but I'm having trouble implementing. Basically, I wanted to know what the easiest way to do this was. I have the external model and the optimization algorithms.

So far, I have used Jupyter Notebooks for my Brightway2 models, but I often get errors when I convert the notebooks to python modules and run them in the Brightway2 environment in IPython. Is there a reason the modules should run differently in IPython than in Jupyter Notebooks?

I was thinking of using PyAutoGUI to send inputs to the Brightway2 environment and IPython. Is there an easier/better way to do that?

Is there a way to import the necessary Brightway modules without running in the Brightway2 environment?

Thanks

Here's an example of an error I get with IPython, but not with Jupyter notes. When I run the following code in Jupyter notes it runs fine.

from brightway2 import *

def main():
    project_name = "Algae_LCA"
    projects.set_current(project_name)
    bw2setup()
    methods.load()

    #Set directory for Ecoinvent v3.2 datasets and name the database.
    data_directory =  "E:\GOOGLE~1\ECOINV~1\ECOINV~1.2-C\datasets"
    database_name = "Ecoinvent_v3.2-Conseq"


    #Import the database, apply cleaning strategies, and provide statistics  
    ei = SingleOutputEcospold2Importer(data_directory, database_name)
    ei.apply_strategies()
    ei.statistics()

But if I run it in IPython in the bw2 environment, it hangs up/crashed on

 ei = SingleOutputEcospold2Importer(data_directory, database_name)

It gives me the following error:

-------------------------------------------------------------
AttributeError                            Traceback (most rec
C:\bw2-python\Algae LCA\BW2_Project_Database_Setup_Test.py in
 36
 37 if __name__ == "__main__":
 ---> 38     main()
 39

C:\bw2-python\Algae LCA\BW2_Project_Database_Setup_Test.py in
 25     #Import the database, apply cleaning strategies,
 26
 ---> 27 ei = SingleOutputEcospold2Importer(data_directory
 28      #ei.apply_strategies()
 29      #ei.statistics()

 C:\bw2-python\envs\bw2\lib\site-packages\bw2io\importers\ecos
 47
 48         start = time()
 ---> 49    self.data = Ecospold2DataExtractor.extract(di
 50         print(u"Extracted {} datasets in {:.2f} secon
 51             len(self.data), time() - start))

 C:\bw2-python\envs\bw2\lib\site-packages\bw2io\extractors\eco
 77
 78         if use_mp:
 ---> 79            with multiprocessing.Pool(processes=multi
 80                 print("Extracting XML data from {} da
 81                 results = [

C:\bw2-python\envs\bw2\lib\multiprocessing\context.py in Pool
116         from .pool import Pool
117         return Pool(processes, initializer, initargs,
--> 118               context=self.get_context())
119
120     def RawValue(self, typecode_or_type, *args):

C:\bw2-python\envs\bw2\lib\multiprocessing\pool.py in __init_
166         self._processes = processes
167         self._pool = []
--> 168     self._repopulate_pool()
169
170         self._worker_handler = threading.Thread(

C:\bw2-python\envs\bw2\lib\multiprocessing\pool.py in _repopu
231             w.name = w.name.replace('Process', 'PoolW
232             w.daemon = True
--> 233         w.start()
234             util.debug('added worker')
235

C:\bw2-python\envs\bw2\lib\multiprocessing\process.py in star
103                'daemonic processes are not allowed to
104         _cleanup()
--> 105     self._popen = self._Popen(self)
106         self._sentinel = self._popen.sentinel
107         _children.add(self)

C:\bw2-python\envs\bw2\lib\multiprocessing\context.py in _Pop
311         def _Popen(process_obj):
312             from .popen_spawn_win32 import Popen
--> 313         return Popen(process_obj)
314
315     class SpawnContext(BaseContext):

 C:\bw2-python\envs\bw2\lib\multiprocessing\popen_spawn_win32.
 32
 33     def __init__(self, process_obj):
 ---> 34    prep_data = spawn.get_preparation_data(proces
 35
 36         # read end of pipe will be "stolen" by the ch

C:\bw2-python\envs\bw2\lib\multiprocessing\spawn.py in get_pr
171     # or through direct execution (or to leave it alo
172     main_module = sys.modules['__main__']
--> 173 main_mod_name = getattr(main_module.__spec__, "na
174     if main_mod_name is not None:
175         d['init_main_from_name'] = main_mod_name

AttributeError: module 'main' has no attribute 'spec'

Answer 1

The problem you are running into is that multiprocessing doesn't work in the (i)python shell on Windows. The notebook avoids this problem through, basically, magic. The Ecospold2DataExtractor uses multiprocessing by default to speed up the extraction of many Ecospold2 files. This should probably be made optional; for now, you can do one of the following:

1. You should only need to import ecoinvent 3.2 once, so this could be done in a) a notebook, or b) a separate python script that you invoke on the command line.
2. Use the hack of writing an import script and then importing that, instead of doing in the import directly in the python session (see the SO link above for more detail).

In response to some other questions/concerns:

Is there a reason the modules should run differently in IPython than in Jupyter Notebooks?

No. Any time this happens should be reported as a bug.

I was thinking of using PyAutoGUI to send inputs to the Brightway2 environment and IPython. Is there an easier/better way to do that?

GUIs are hard - you are welcome to write one!

Is there a way to import the necessary Brightway modules without running in the Brightway2 environment?

There is no Brightway2 environment - just a set of python packages that can be imported. You can import them separately (though some depend on each other), e.g. bw2calc can be run independent of everything else.