6

I have a script in Python which do some computations. When I run this script in console it takes about 7 minutes to complete but when I run it thought Java shell it takes three times longer. I use following code to execute the script in Java:

this.p = Runtime.getRuntime().exec("script.py --batch", envp);

this.input = new BufferedReader(new InputStreamReader(p.getInputStream()));
this.output = new BufferedWriter(new OutputStreamWriter(p.getOutputStream()));
this.error = new BufferedReader(new InputStreamReader(p.getErrorStream()));

Do you have any suggestion why the Python script runs three time longer in Java than in a console?

update (29.12.2010)

The computation goes as follow:

  1. Java sends data to the Python.
  2. Python reads the data.
  3. Python generates a decision tree --- this is a long operation.
  4. Python sends a confirmation that the tree is ready.
  5. Java receives the confirmation.

Later there is a series of communications between Java and Python but it takes only several second.

update (29.12.2010)

Thank you for all your comments and suggestions. It took one working day to find out that my assumption was wrong. The code I used had 'a bug' and in fact different computation were performed in console and in shell. When I fixed it the computation time was the same.

Summary: The computation time of a script run in console and in Java shell is almost the same. The additional time for initialize Java VM and IO communication is insignificant.

czuk
  • 6,218
  • 10
  • 36
  • 47
  • 2
    Is there a lot of IO? Or is it a CPU-bound process? – moinudin Dec 28 '10 at 11:34
  • It is CPU-bound process. The script receives data only at the beginning but this is done quickly. After that the script creates a decision tree what is very time-consuming. – czuk Dec 28 '10 at 11:41
  • What are you doing with `input` and `error`? Throwing it away or keeping it in memory? And how much output does you program generate to these streams? – rodion Dec 28 '10 at 11:42
  • Do you take the startup time of the Java runtime into account? That can take a few seconds. – extraneon Dec 28 '10 at 11:44
  • @extraneon: Yes, but it takes only several seconds and comparing to the total time of processing is not significant. – czuk Dec 28 '10 at 11:52
  • Are you sure you are properly handling streams? Flushing when needed etc.? – Rekin Dec 28 '10 at 11:52
  • Is your machine thrashing while the script is running? Is the script writing a lot of stuff to its stdout / stderr? – Stephen C Dec 28 '10 at 11:52
  • The script works fine. It completes the processing without any visible errors. The only problem is that the time is three times longer. I think I flush the streams were needed, in other way the communication between Java and Python would fail. – czuk Dec 28 '10 at 12:09
  • Perhaps try a different method of executing the script. Dump its input into a file, pipe that file to it and pipe its output to file and read that file once it's done. – moinudin Dec 28 '10 at 12:15
  • 1
    Are you doing `p.waitFor()`? Can you clarify how step 4 is implemented, please. – rodion Dec 28 '10 at 12:33
  • 1
    @czuk: can you try, just to see, how long it takes in both cases if you redirect all the outputs (stdout and stderr) to */dev/null* ? – SyntaxT3rr0r Dec 28 '10 at 14:02

1 Answers1

2

Try using the same code without the BufferedReaders and BufferedWriters, in case there is delay introduced by the buffering. I'm not sure how long buffered writers wait to flush, but at the very least, removing them would help simplify your problem.

James Branigan
  • 1,164
  • 7
  • 9