-2

I have three python files file1.py, file2.py, file3.py. Each file will generate a .csv file and will give it to the other file sequentially. To elaborate file1.py will generate file1.csv and this .csv file will be the input of file2.py and so on. 

 import file1
 import file2
 import file3

 file1
 file2
 file3

 IOError: File file2.csv does not exist

The problem is when I import file2.py,there is no file1.csv as it is yet to be executed. Please let me how to call each one sequentially without even reading the next python file.

Caleb Kleveter
  • 11,170
  • 8
  • 62
  • 92
Manu9
  • 31
  • 1
  • 7
  • You don't have to import everything at the top. You can import, run, import, run. – salezica Oct 02 '14 at 20:09
  • @uʍop ǝpısdn I even tried that but the python will first read all the import files and then it will run the files. – Manu9 Oct 02 '14 at 20:11
  • `but the python will first read all the import files and then it will run the files` <- not always – inspectorG4dget Oct 02 '14 at 20:12
  • 2
    It looks like you need some serious refactoring; your scripts shouldn't generally do anything when imported. – jonrsharpe Oct 02 '14 at 20:14
  • @jonrsharpe I am using Canopy as IDE. I am getting errors while import. Will subroutine work? I don't know how to use subroutines. – Manu9 Oct 02 '14 at 20:20
  • 1
    I think I have a better idea of what's going on now.... file1.py, etc... should define functions that take a csv file as input and return a csv file as output. Import all the modules at the top and then call the functions one by one. – tdelaney Oct 02 '14 at 20:29

3 Answers3

2

You can pipeline from the command line

python file1.py|python file2.py|python file3.py

The files would use sys.stdin / sys.stdout as their csv readers / writers.

EDIT

You can write a controller script that fires them all off. sys.executable is just the python used to start the script... I'm using it so that the same python is used for all of them.

import sys
import subprocess as subp

file1 = subp.Popen([sys.executable, 'file1.py'], stdout=subp.PIPE)
file2 = subp.Popen([sys.executable, 'file2.py'], stdin=file1.stdout, stdout=subp.PIPE)
file3 = subp.Popen([sys.executable, 'file3.py'], stdin=file2.stdout)

# close parent version of pipes
file1.stdout.close()
file2.stdout.close()

# wait for programs to complete
file1.wait()
file2.wait()
file3.wait()
tdelaney
  • 73,364
  • 6
  • 83
  • 116
2

Each of your scripts should be set up as a function or class that generates the file. The general structure could be something like this:

# file1.py
def generate_csv(filename):
    # put your file generation code here
    # you could easily use Python's csv module, for example
    return csv_filename

Then in your main script you can call each of your scripts, like this:

# main.py
import file1, file2, file3

def main(filename):
    fname_one = file1.generate_csv(filename)
    fname_two = file2.generate_csv(fname_one)
    fname_three = file3_generate_csv(fname_two)

This keeps your original scripts from being run when imported. Your main script controls the order of execution and can do whatever needs to be done to the 3rd file name that is returned.

Mike Driscoll
  • 32,629
  • 8
  • 45
  • 88
  • I am getting the following error ValueError: No columns to parse from file – Manu9 Oct 02 '14 at 20:59
  • You should be able to use whatever code you already have and just indent it inside of the function. And then return the path of the file it created. – Mike Driscoll Oct 02 '14 at 21:01
0

If you expect to read from sys.argv and output to stdout in such a setup should work:

$ python file3.py $(python file2.py $(python file1.py))

Note: The leading $ denotes this is a bash shell command.

Though BE CAUTIOUS. I typically will $ echo $(python file1.py) before passing it to the second script, et cetera.

ThorSummoner
  • 16,657
  • 15
  • 135
  • 147