How to call UNIX sort command on data in pipe

Question

I am creating a C program and with it I am setting up a pipe between separately forked process for interprocess communication.

The first process has written the data I need into the pipe. However, with the second process reading from the pipe, I am trying to exec the process to become the UNIX sort command. I want to somehow call sort on the data in the pipe.

How can I call sort on a pipe? On the commandline, I can sort by supplying the filename to sort as a commandline argument e.g. "sort -r MyFileToSort". I know that pipes are essentially considered files, but they are only described by their file descriptor, and as far as I know, sort won't know what to do with a fd.

Thanks for any help/feedback

@martinclayton, why would a named pipe be necessary here? One can simply attach FDs returned by `mkpipe()` to stdin or stdout, as necessary. — Charles Duffy, Jan 07 '15 at 00:43
@iharob I've tried creating a file and writing data from the pipe to that file. Then I call sort on that file via an exec call from a child process and output to the file. This works, but doesn't give quite what I want. The newly created file first displays all the pipe content, and then displays a duplicate of the pipe content except this time sorted. I essentially want the last output file to only have the sorted pipe content. On the commandline it would look like: "data | sort > MyOutputFile" I want data piped into the sort command and then have that output redirected into MyOutputFile. — pleaver, Jan 07 '15 at 00:46
@martinclayton, sure, but I'm not sure how it helps -- unless you feel like increasing filesystem churn (generating, and presumably later deleting, directory entries) for its own sake. If there's any way in which using named pipes provides a material benefit over unnamed ones for this use case, I can't think of it. — Charles Duffy, Jan 07 '15 at 00:48
@pleaver, ...by the way -- if you're not clear how one would set this up, you might consider reading through the output of `strace bash -c 'generate_data | sort >MyOutputFile'`, to see how the shell is accomplishing the task. — Charles Duffy, Jan 07 '15 at 00:51
@JonathanLeffler Sorry I'm a new user. I really appreciate the rapid help provided. I wish I could upvote but I dont have the rep yet. I'll choose your answer due to code example but I also appreciate the answer of Charles Duffy — pleaver, Jan 07 '15 at 01:17

Charles Duffy · Answer 1 · 2015-01-07T00:53:25.827

0

You don't need to pass sort any arguments to specify input source or output sink at all in this case. Instead, before execing it, you should make attach your pipeline's file descriptors to its stdin (FD 0, if receiving data from a pipe) or stdout (FD 1, if writing data to a pipe), as appropriate.

See the dup2() call, which lets you set the destination to which you're copying a FD, for this purpose. As @JonathanLeffler points out, you'll want to be sure to close the original FDs (after duplicating them to the numbers you want) before your exec call.

Since you've clarified, in comments, that your goal is to write to a file, you would attach FD 1 to that destination file before calling exec, with FD 0 attached to the output side of the pipeline containing input.

edited Jan 07 '15 at 00:53

answered Jan 07 '15 at 00:45

Charles Duffy

280,126
43
390
441

1

And close the unused file descriptors before the `exec*()` call -- both the file descriptors from `pipe()` in this case. – Jonathan Leffler Jan 07 '15 at 00:46
Thanks, I redirected standard input to the reader of the pipe and now calling sort with no file specified works fine. I'm still having a slight issue, but the context of this question has been answered – pleaver Jan 07 '15 at 01:01

score 0 · Accepted Answer · answered Jan 07 '15 at 00:59

int p[2];
if (pipe(p) != 0) ...report error and do not continue...
pid_t pid = fork();
if (pid < 0) ...report error, close pipe descriptors, and do not continue...
if (pid == 0)
{
    /* Child - becomes sort */
    dup2(p[0], 0);
    close(p[0]);
    close(p[1]);
    int fd = open("output-file", O_CREAT | O_EXCL | O_WRONLY, 0644);
    if (fd < 0) ...report error and exit...
    dup2(fd, 1);
    close(fd);
    execlp("sort", "sort", (char *)0);
    ...report error and exit...
}
else
{
    /* Parent - writes data to sort */
    close(fd[0]);
    ...write data to fd[1]...
    close(fd[1]);
    int status;
    int corpse;
    while ((corpse = wait(&status)) > 0 && corpse != pid)
        ...consider reporting which child died...
    ...consider reporting sort status...
    ...continue with the rest of the program...
}

You can decide whether to report errors related to dup2() failing, or close() failing. There isn't much you can do in either case except report the problem and exit. Unless someone has subjected your program to cruel and unusual punishment by not supplying it with standard input, standard output and standard error (or something elsewhere in the program has closed any of the standard channels), then the pipe and file descriptors can't be the standard I/O descriptors, so the closes are safe. If you're not sure how sick your users are, you might protect the closes:

if (p[0] > FILENO_STDERR)
    close(p[0]);

That is normally unnecessarily paranoid (but it can be fun trying programs with missing standard I/O).

Thank you for the code example. I thought that sort had to be explicitly passed a file name, but I see that redirecting stdin to the pipe and just calling sort works — pleaver, Jan 07 '15 at 01:03
Many, but by no means all, Unix programs work as 'filters' and read standard input if given no file names and write to standard output. You could use: `execlp("sort", "sort", "-o", "output-file", (char *)0);` instead of doing the I/O redirection in your code. That causes `sort` to write to `output-file`. In general, it allows `sort` to write to one of its input files; you're just using the option as a side-effect if you adopt the change. If you might have sort options, you should consider `execvp()` or `execv()` instead of `execlp()`; you can have variable length argument lists that way. — Jonathan Leffler, Jan 07 '15 at 01:06

How to call UNIX sort command on data in pipe

2 Answers2

Linked