2

I am looking for a JAVA library that facilitates running external programs. Now I run them "manually" in a separate thread and capture I/O.

I have several different external tools to run and what I need is a uniform approach to do that. The tasks I have to handle includes: - preparing input files according to predefined templates - running the commands - waiting for the results and parsing output files - maintaining the pipeline

The whole pipeline can be considered as a graph with external tools as nodes and data flow as edges. It would be great if the software could run some of the commands in parallel threads, if the data flow permits.

Is there an existing solution of such problems?

Based on the answers I got I feel I have to clarify: I don't need pipes. Data flow can be solved with files, which I need anyway. Moreover, pipes have to be linear (1 output -> 1 input) but I need a graph

I already have a kind of a prototype in python - a bunch of scripts. It is good but as for me - not scalable. Moreover some of the programs I call are in java, so making the whole thing in java would be handy. Best, Tim

tnorgd
  • 1,580
  • 2
  • 14
  • 24
  • Do you want to pipe stdout from one program to stdin of another? I doubt if this is possible in Java. As for concurrency: this is simple, create one Thread per external processes and just do Process.waitFor for all programs the new one depends on. Or better, let a Thread wait on the process it started and check what processes become eligible. – maaartinus Jan 22 '11 at 11:51
  • 1
    You can use ProcessBuilder to run an external program. To pipe data from one program to another, use your shell with `|` between the two prorgams. – Peter Lawrey Jan 22 '11 at 12:07

2 Answers2

1

If you're on Unix, have you thought about building a Unix shell command line dynamically (using pipes, redirection, tee etc.) and then spawning this one command off using ProcessBuilder or similar, and (say) /bin/sh -c ?

It means you're making use of an existing working infrastructure that handles pipes, buffering, error collection and managing resources.

Brian Agnew
  • 268,207
  • 37
  • 334
  • 440
1

I would think ant should be perfect for this. It also has fairly good java library to do file processing. Between that and maybe velocity templates, i would think you should be able to do what you want.

gbvb
  • 866
  • 5
  • 10
  • Yes, ant would be the best option and I've considered it already. But it has a few serious problems: (1) can I declare variables and arrays? It often happens that a result from one program is an array of strings (e.g. other commands) that is used by other executables – tnorgd Jan 22 '11 at 17:40
  • I think ant-contrib has support for variables. http://ant-contrib.sourceforge.net/tasks/tasks/variable_task.html – gbvb Jan 22 '11 at 17:48