Why can't get data from tail -f?

Question

In my program I'm redirecting output of child process to pipe and in parent getting this result and doing something (this is not important). But my program doesn't work when I'm using tail -f test.txt command and doesn't recieve any data during tail is running, and getting this data only after tail is finished (or killed).

At first I have thought that problem was that tail -f doesn't flushing and that's why no data I can recieve, but when I have tried to redirect output of tail -f to some file the data were in this file even when tail were not finished.

//the code of creating child and redirecting data (important part)
//only core is here so please don't tell me that maybe pipe() or fork() is failed

pid_t pid;
int outpipe[2]; //pipe for reading from stdout
int errpipe[2]; //pipe for reading from stderr

// Createing pipes for childs stdout and stderr streams
pipe(outpipe);
pipe(errpipe);
pid = fork();
if(pid == 0)
{
    // This is the child process. Closing read end of pipes and duplicating stdout and stderr streams
    close(outpipe[0]);
    dup2(outpipe[1], STDOUT_FILENO);
    close(errpipe[0]);
    dup2(errpipe[1], STDERR_FILENO);

    if(execvp(argv[0], (char * const *)argv) == -1)
    {
        fprintf(stderr, "Failed to execute command %s: %s", argv[0], strerror(errno));
        _exit(EXIT_FAILURE);
    }

    _exit(EXIT_SUCCESS);
}
else if (pid != -1)
{
    // This is the parent process, Closing write end of pipes and opening fds as FILE
    close(outpipe[1]);
    *child_stdout_stream=fdopen(outpipe[0], "rt");
    close(errpipe[1]);
    *child_stderr_stream=fdopen(errpipe[0], "rt");
    *child_pid=pid;
}

Then I'm reading from child_stderr_stream and child_stdout_stream which were passed as parameters to function the part above is from what. For reading I'm using select() to not block program until reading from one of the streams.

Adding part of select and read

int select_and_read(FILE **files, bool *is_eof, char *chars, int *mask, int nfiles, int timeout, pid_t child_pid)
{
    int max_fd_plus_1 = 0;
    fd_set rfds;
    struct timeval tv;

    FD_ZERO(&rfds);

    for(int i = 0; i < nfiles; ++i)
    {
        if(is_eof[i]==false)
        {
            FD_SET(fileno(files[i]), &rfds);
            max_fd_plus_1 = (max_fd_plus_1 > fileno(files[i])) ? max_fd_plus_1 : fileno(files[i]);
        }
    }
    ++max_fd_plus_1;

    tv.tv_sec = timeout / 1000;
    tv.tv_usec = (timeout % 1000) * 1000;

    int retval = select(max_fd_plus_1, &rfds, NULL, NULL, &tv);
    if(retval > 0)
    {
        *mask = 0;
        for(int i = 0; i < nfiles; ++i)
        {
            if(is_eof[i]==false)
            {
                if(FD_ISSET(fileno(files[i]), &rfds))
                {
                    *mask |= 1 << i;
                    chars[i] = fgetc(files[i]);
                }
            }
        }
    }
    else
    {
        kill(child_pid, SIGKILL);
    }
    return retval;
}

problem could be in select(). Add some debugging to see if the FD is ready when you attach to stdout. Perhaps you are select() on stdout? — FoneyOp, Apr 15 '11 at 15:35
Yes select() doesn't tells me that I can read from stream. It hangs, but why? How can I solve this problem? — Mihran Hovsepyan, Apr 15 '11 at 15:37
What are you select()ing on ? Seems like there is some important parts missing here. — FoneyOp, Apr 15 '11 at 15:39
Instead of forking a process to do it, find the source for `tail`, learn how it works and do it yourself. Once you peel out all of the argument handling, there isn't much to it. I did the same thing many years ago to write the equivalent of doing `cat` followed by `tail -f` and it wasn't a big effort. — Blrfl, Apr 15 '11 at 15:42
@Blrfl may program is not alternative to tail, it just can run it. — Mihran Hovsepyan, Apr 15 '11 at 15:45
first, you should know what max_fd_plus_1 is with tail and without tail. Also what is the retval for select() in both cases? Seems like the problem is in getting the correct FD and the call to select(). Also if you are doing async IO, do you really need to fork()? You application might be simpler with single threaded ASYNC or multi threaded blocking IO. — FoneyOp, Apr 15 '11 at 15:58
To kind of echo FoneyOp, I think I would solve this using popen and fgets in a separate thread. It does a fair porption of the pipe work for you. Something in the same vein as http://cplusplus.com/forum/beginner/28982/ — Robb, Apr 15 '11 at 16:26
Yes popen works correct, but I need select to read from both file descriptors (stdout and stderr). — Mihran Hovsepyan, Apr 15 '11 at 17:03

score 1 · Accepted Answer · answered Apr 17 '11 at 09:30

1

This strange problem have been solved very strangely. I have just set buffers of files to 0 this way:

else if (pid != -1)
{
    // This is the parent process, Closing write end of pipes and opening fds as FILE
    close(outpipe[1]);
    *child_stdout_stream=fdopen(outpipe[0], "rt");
setbuf(*child_stdout_stream, NULL);
    close(errpipe[1]);
    *child_stderr_stream=fdopen(errpipe[0], "rt");
setbuf(*child_stderr_stream, NULL);
    *child_pid=pid;
}

This is very strange, that this helps, but in any case my program is now working well.

answered Apr 17 '11 at 09:30

Mihran Hovsepyan

10,810
14
61
111

This isn't strange, This is just a buffering problem. Had your program printed more data in your stream, then you would see that you would get the output every X bytes (where X is often 4096 but not always), which is good for performance and battery life (your program sleeps more). But if you want to see lines when they arrive, using line buffering would be better than using no buffering. – BatchyX Apr 17 '11 at 09:53
This is strange beacause problem was only with tail. Other tail kind programs worked well. – Mihran Hovsepyan Apr 17 '11 at 10:31

Why can't get data from tail -f?

1 Answers1

Linked