2

TL;DR: Is it posixly incorrect for fflush(NULL) not to flush stdin?

Consider the following code snippet,

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void) {
    char line[128] = {0};
    fgets(line, sizeof(line), stdin);
    puts(line);
    fflush(NULL);
    pid_t pid = fork();
    if (pid == 0) {
        fgets(line, sizeof(line), stdin);
        puts(line);
        exit(EXIT_SUCCESS);
    } else {
        waitpid(pid, NULL, 0);
        fgets(line, sizeof(line), stdin);
        puts(line);
    }
    return 0;
}

When fed a file with 3 lines using file redirection, it exhibits different behaviour on macOS and Linux,

$ cat test
line 1
line 2
line 3
macOS $ ./a.out < test
line 1

line 2

line 3

Linux $ ./a.out < test
line 1

line 2

line 2

The reason seems to be that glibc fflush(NULL) does not flush stdin although fflush(stdin) is well defined. glibc manual reads,

If stream is a null pointer, then fflush causes buffered output on all open output streams to be flushed.

while POSIX states,

For a stream open for reading with an underlying file description, if the file is not already at EOF, and the file is one capable of seeking, the file offset of the underlying open file description shall be set to the file position of the stream, and any characters pushed back onto the stream by ungetc() or ungetwc() that have not subsequently been read from the stream shall be discarded (without further changing the file offset).

If stream is a null pointer, fflush() shall perform this flushing action on all streams for which the behavior is defined above.

macOS libc source glibc source musl source

I am suspecting it to be a non-compliance bug in glibc/musl, but considering the enormous install base, I wonder if it is indeed against POSIX?

ikegami
  • 367,544
  • 15
  • 269
  • 518
SuibianP
  • 99
  • 1
  • 11
  • 1
    The point you miss in the POSIX quote is "and the file is one capable of seeking"... Standard input is generally not capable of seeking. Once you have read a character from standard input, it's not possible to seek back and retrieve it later. – Some programmer dude May 09 '23 at 16:36
  • 3
    @Some programmer dude, While it's true that stdin isn't seekable in general, it is seekable in this particular instance. – ikegami May 09 '23 at 17:00
  • 1
    I found `POSIX.1-2001 did not specify the behavior for flushing of input streams, but the behavior is specified in POSIX.1-2008`. Indeed https://pubs.opengroup.org/onlinepubs/009695399/ – KamilCuk May 09 '23 at 17:08
  • Is the behavior different when you omit the `fflush`? You call `fflush` before `fork`, so I would assume that both processes have the same position in their `stdin` stream, and I would rather expect the Linux behavior. Is there a specification that explains the MacOS behavior for the case that `stdin` is redirected from a regular file? – Bodo May 09 '23 at 17:08
  • Additional info: Running the program under `strace` reveals that `fflush( NULL );` does not perform a seek for `stdin`. However, a seek can be observed when changing `fflush( NULL )` to `fflush( stdin );`, and the output changes to the output observed in MacOS. – ikegami May 09 '23 at 17:09
  • @Bobo, Re "*Is the behavior different when you omit the `fflush`?*", Linux: No. MacOS: Not tested, but I expect you 1,2,2. – ikegami May 09 '23 at 17:11
  • 1
    What if the file is already at EOF? For example, your test file looks like it has 18 chars. That will be read in one read, positioning the fd at EOF. Fflushing an input stream has always been pointless, POSIX's recent apology not withstanding . – mevets May 09 '23 at 17:12
  • @mevets, `stdin` is not at EOF. The underlying fd is, but that doesn't matter. If you were right, `fflush( stdin )` wouldn't work. But it does. – ikegami May 09 '23 at 17:13
  • 1
    That is typically called *happens to work*; and the awkward POSIX reference to file description is "descriptor", as in fd. – mevets May 09 '23 at 17:15
  • 1
    Found https://www.austingroupbugs.net/view.php?id=816 – KamilCuk May 09 '23 at 17:15
  • @mevets, It doesn't say file description, it says file. ("if the file is not already at EOF") The file is not at EOF. Again, if you were right, it would simply mean that `fflush( stdin );` is broken instead of `fflush( NULL )`. They currently do different things, even though they should do the same thing. – ikegami May 09 '23 at 17:15
  • Also from the POSIX rationale section `Data buffered by the system may make determining the validity of the position of the current file descriptor impractical. Thus, enforcing the repositioning of the file descriptor after fflush() on streams open for read() is not mandated by POSIX.1-2017.` – KamilCuk May 09 '23 at 17:23
  • @KamilCuk, and that rationale reflects the order of the paragraphs in the description of `fflush()` in POSIX.1-2008. Oddly, however, the paragraph order has changed in POSIX.1-2017, yet the same rationale is present. – John Bollinger May 09 '23 at 18:20

1 Answers1

0

TL;DR: Is it posixly incorrect for fflush(NULL) not to flush stdin?

You have already quoted the relevant POSIX provisions.

So, with respect to your example,

  • Is stdin a stream open for reading? Yes.
  • Does it have an underlying open file description? It should, since you're redirecting from a regular file.
  • Is that file capable of seeking? It should be, since you're redirecting from a regular file.
  • Is the file already at EOF? Here's the rub. It says "the file" not "the stream". When the standard input is connected to a regular file, it defaults to being fully buffered, and that buffer is probably large enough to accommodate your full example input. Thus, in your example program, it is likely that after the first read from stdin, the underlying file is at EOF, in which case POSIX does not specify any behavior for fflushing it.

More generally, then, there are conditions under which POSIX does not mandate that fflush(NULL) should flush stdin:

  • if it does not have an underlying open file description (though I'm unsure under what circumstances that would happen)

  • if the underlying file is incapable of seeking -- this is typically the case when stdin is connected to a terminal, for example

  • if the underlying file has already reached EOF

On the other hand, if Glibc's fflush(NULL) fails to flush stdin when all of the POSIX criteria are satisfied, then yes, that is posixly incorrect.

John Bollinger
  • 160,171
  • 8
  • 81
  • 157
  • On "Is the file already at EOF?" you seem to indicate it should matter if the raw file descriptor is at EOF. It doesn't seem like that should matter; if I read 1 byte from the file, and that causes it to buffer the rest of the file data, the only "file" it seems like they'd be referring to is the `FILE*`, which is not at EOF. Seems like the only reason for `fflush(stdin)` is to reposition the fd and discard all buffered data (losing `ungetc` effects because that's defined by buffer, not fd seek position), and having the real fd's position matter messes that up. – ShadowRanger May 09 '23 at 17:36
  • @ShadowRanger, I acknowledge that potential difference in interpretation. POSIX is pretty consistent about terminology, however, and notwithstanding the type's name, the object represented by a `FILE` is called a *stream*, not a file. Moreover, do you propose that it is talking about a different "file" in "the file is not already at EOF" than it is in the very next clause of the same sentence, "the file is one capable of seeking"? – John Bollinger May 09 '23 at 17:51
  • Hmm... I was unaware of the "stream" vs. "file" nomenclature. Not sure why the seeking clause is relevant to interpretation though; a `FILE*` can be capable, or incapable, of seeking, even if it's only because the underlying descriptor's capabilities. I do acknowledge that the use of "stream" early on seems to imply that "file" is something different in all other uses. It still seems like a standard defect to make the implementation details of the buffering and the position of the underlying file handle matter for behavior though. – ShadowRanger May 09 '23 at 18:24
  • @ShadowRanger, I think there is a defect here at least inasmuch as the standard is open to multiple interpretations. Possibly also with respect to whether `fflush(NULL)` is supposed to apply to input streams -- it did not in POSIX.1-2008, and the spec contains a rationale that seems to apply to that. A reordering of the description changes that in in POSIX.1-2017, yet the rationale remains. And yes, it may well be that the requirement was intended to be that the stream not have reached EOF. – John Bollinger May 09 '23 at 18:36