2

I have program I cant modify, as is, and I need to execute it, write some data to its stdin and get the answer from its stdout in programmatic manner, automated. What is the simpliest way to do this?

I suppose something like this pseudo-C-code

char input_data_buffer[] = "calculate 2 + 2\nsay 'hello world!'";
char output_data_buffer[MAX_BUF];
IPCStream ipcs = executeIPC("./myprogram", "rw");
ipcs.write(input_data_buffer);
ipcs.read(output_data_buffer);
...

PS: I thought of popen, but AFAIK there is only one-way pipes in linux

EDIT:

It is supposed it will be one-message-from-each-side communication. Firstly parent side send input to child process' stdin, then child provides output to its stdout and exits, meanwhile parent reads its stdout. Now about communication termination: I think when child process exits it will send EOF terminator to stdout, so parent will know exactly whether child done, on the other hand it is guaranteed that parent knows what kind of input child expects for.

Generally this program (parent) - a student's solution tester. It takes paths to two other executables from CLI, the first is student's program to test, the second is etalon correctly working program, which solves very same problem.

Input/output of students programs is strictly specified, so tester run both programs and compares its outputs for lots of random inputs, all mismatches will be reported.

Input/output max size is estimated at few hundreds kilobytes

Example: ..implement insertion sort algorithm ... first line there is sequence length ... second line there is sequence of numbers a_i where |a_i| < 2^31 - 1... output first line must be sum of all elements, the second line must be sorted sequence.

Input:

5
1 3 4 6 2

Expected output:

16
1 2 3 4 6
kernelbug
  • 151
  • 1
  • 2
  • 13
  • 2
    How big are the messages going to the other program? And the responses? Will you be sending one message each way, or will it be an ongoing conversation? These issues matter because both programs need to know when they've reached the end of the current input message. A simple system sending one short request with a one short response is easy to code (short being less than about 4 KiB). As the messages get longer, it is trickier to synchronizate the two programs. You'll probably be using a 'half duplex' protocol; one process is sending and the other is receiving at all times — that's easier too. – Jonathan Leffler Oct 26 '13 at 05:42
  • Check out my edit please. Maybe, is it possible to use 'half duplex' at first to one side then to another side? – kernelbug Oct 26 '13 at 08:44

3 Answers3

4

Read Advanced Linux Programming -which has at least an entire chapter to answer your question- and learn more about execve(2), fork(2), waitpid(2), pipe(2), dup2(2), poll(2) ...

Notice that you'll need (at least in a single-threaded program) to multiplex (with poll) on the input and the output of the program. Otherwise you might have a deadlock: the child process could be blocked writing to your program (because the output pipe is full), and your program could be blocked reading from it (because the input pipe is empty).

BTW, if your program has some event loop it might help (and actually poll is providing the basis for a simple event loop). And Glib (from GTK) provides function to spawn processes, Qt has QProcess, libevent knows them, etc.

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • okay, looks like there is no best practice pattern to do such simple thing and i have to go deeper into linux IPC coding. Now im pretty hazy understand how to do that so I will take your advice and learn about poll and this plenty of kernel funcs. And no, there is no event loop, just function-like *start-input-output-end* communication. – kernelbug Oct 25 '13 at 21:57
2

Given that the processing is simply a question of one message from parent to child (which must be complete before the child responds), and one message from child to parent, then it is easy enough to handle:

  • Create two pipes, one for communication to child, one for communication to parent.
  • Fork.
  • Child process duplicates the relevant ends of the pipes (read end of 'to-child' pipe, write end of 'to-parent' pipe) to standard input, output.
  • Child closes all pipe file descriptors.
  • Child execs test program (or prints a message to standard error reporting failure and exits).
  • Parent closes the irrelevant ends of the pipes.
  • Parent writes the message to the child and closes the pipe.
  • Parent reads the response from the child and closes the pipe.
  • Parent continues on its merry way.

This leaves the child process lying around as a zombie. If the parent is going to do this more than once, or just needs to know the exit status of the child, then after closing the read pipe, it will wait for the child to die, collecting its status.

All this is straight-forward, routine coding. I'm sure you could find examples on SO.


Since apparently there are no suitable examples on Stack Overflow, here is a simple implementation of the code outlined above. There are two source files, basic_pipe.c for the basic piping work, and myprogram.c which is supposed to respond to the prompts shown in the question. The first is almost general purpose; it should probably loop on the read operation (but that hasn't mattered on the machine I tested it on, which is running an Ubuntu 14.04 derivative). The second is very specialized.

System calls

basic_pipe.c

#include <assert.h>
#include <errno.h>
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>

static char msg_for_child[] = "calculate 2 + 2\nsay 'hello world!'\n";
static char cmd_for_child[] = "./myprogram";

static void err_syserr(const char *fmt, ...);
static void be_childish(int to_child[2], int fr_child[2]);
static void be_parental(int to_child[2], int fr_child[2], int pid);

int main(void)
{
  int to_child[2];
  int fr_child[2];
  if (pipe(to_child) != 0 || pipe(fr_child) != 0)
    err_syserr("Failed to open pipes\n");
  assert(to_child[0] > STDERR_FILENO && to_child[1] > STDERR_FILENO &&
         fr_child[0] > STDERR_FILENO && fr_child[1] > STDERR_FILENO);
  int pid;
  if ((pid = fork()) < 0)
    err_syserr("Failed to fork\n");
  if (pid == 0)
    be_childish(to_child, fr_child);
  else
    be_parental(to_child, fr_child, pid);
  printf("Process %d continues and exits\n", (int)getpid());
  return 0;
}

static void be_childish(int to_child[2], int fr_child[2])
{
  printf("Child PID: %d\n", (int)getpid());
  fflush(0);
  if (dup2(to_child[0], STDIN_FILENO) < 0 ||
      dup2(fr_child[1], STDOUT_FILENO) < 0)
    err_syserr("Failed to set standard I/O in child\n");
  close(to_child[0]);
  close(to_child[1]);
  close(fr_child[0]);
  close(fr_child[1]);
  char *args[] = { cmd_for_child, 0 };
  execv(args[0], args);
  err_syserr("Failed to execute %s", args[0]);
  /* NOTREACHED */
}

static void be_parental(int to_child[2], int fr_child[2], int pid)
{
  printf("Parent PID: %d\n", (int)getpid());
  close(to_child[0]);
  close(fr_child[1]);
  int o_len = sizeof(msg_for_child) - 1; // Don't send null byte
  if (write(to_child[1], msg_for_child, o_len) != o_len)
    err_syserr("Failed to write complete message to child\n");
  close(to_child[1]);
  char buffer[4096];
  int nbytes;
  if ((nbytes = read(fr_child[0], buffer, sizeof(buffer))) <= 0)
    err_syserr("Failed to read message from child\n");
  close(fr_child[0]);
  printf("Read: [[%.*s]]\n", nbytes, buffer);
  int corpse;
  int status;
  while ((corpse = waitpid(pid, &status, 0)) != pid && corpse != -1)
    err_syserr("Got pid %d (status 0x%.4X) instead of pid %d\n",
               corpse, status, pid);
  printf("PID %d exited with status 0x%.4X\n", pid, status);
}

static void err_syserr(const char *fmt, ...)
{
  int errnum = errno;
  va_list args;
  va_start(args, fmt);
  vfprintf(stderr, fmt, args);
  va_end(args);
  if (errnum != 0)
    fprintf(stderr, "(%d: %s)\n", errnum, strerror(errnum));
  exit(EXIT_FAILURE);
}

myprogram.c

#include <stdio.h>

int main(void)
{
  char buffer[4096];
  char *response[] =
  {
    "4",
    "hello world!",
  };
  enum { N_RESPONSES = sizeof(response)/sizeof(response[0]) };

  for (int line = 0; fgets(buffer, sizeof(buffer), stdin) != 0; line++)
  {
    fprintf(stderr, "Read line %d: %s", line + 1, buffer);
    if (line < N_RESPONSES)
    {
      printf("%s\n", response[line]);
      fprintf(stderr, "Sent line %d: %s\n", line + 1, response[line]);
    }
  }
  fprintf(stderr, "All done\n");

  return 0;
}

Example output

Note that there is no guarantee that the child will complete before the parent starts executing the be_parental() function.

Child PID: 19538
Read line 1: calculate 2 + 2
Sent line 1: 4
Read line 2: say 'hello world!'
Sent line 2: hello world!
All done
Parent PID: 19536
Read: [[4
hello world!
]]
PID 19538 exited with status 0x0000
Process 19536 continues and exits
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
0

You can use expect to achieve this: http://en.wikipedia.org/wiki/Expect

This is what a usual expect program would do:

# Start the program
spawn <your_program>
# Send data to the program
send "calculate 2 + 2"
# Capture the output
set results $expect_out(buffer)

Expect can be used inside C programs using expect development library, so you can translate previous commands directly into C function calls. Here you have an example:

http://kahimyang.info/kauswagan/code-blogs/1358/using-expect-script-cc-library-to-manage-linux-hosts

You can also use it from perl and python which usually are usually easier to program for these type of purposes than C.

alcachi
  • 598
  • 6
  • 12
  • But this is not C, and the OP asked about C programming! – Basile Starynkevitch Oct 25 '13 at 20:22
  • Right, the good part is that it can be done also from C using expect-dev. Even if I usually prefer perl or python for this type of tasks. – alcachi Oct 25 '13 at 20:35
  • Thank you for your answer, interesting thing, I've never heard of it. But I would like solution only with native system tools, without any auxiliary libs/packets. This one looks like handy for certain situations! – kernelbug Oct 25 '13 at 21:47