3

I'm getting some weird results, while trying to write data to files in C.<br/> I thought that fclose() closes the *FILE and flushes the data from its buffer to the file.<br/> But for some reason it only flushes the data in my program sometimes and it doesn't do it other times.

For example: I would run this code, and in my file I can see the two strings. (Perfect, exactly what I want)
But then when I run the code the next 4 times it doesn't change anything in my file. And then when I run it another time, suddenly the 10 extra strings appear (8 from the last times I ran the program and the 2 from now)
(This 4 times is just an example, sometimes it's 5, 8, 10, or even just 2 times before I see the output appear)

I really don't understand this? Shouldn't the data be visible after every time I run the program? Where is this buffer even saved between the different times I run the program, because the program finishes every time, so the memory gets released, right?

(By the way I also tried fflush(fd) before and after fclose(), but that didn't solve the problem)

#include <stdio.h>


int main(int argc, char const *argv[]) {
  FILE * fd;
  fd = fopen("test_file.txt", "a");
  fprintf(fd, "String 1\n");
  fprintf(fd, "String 2\n");
  fclose(fd);
  return 0;
}
Adam
  • 2,820
  • 1
  • 13
  • 33
piet_lu
  • 77
  • 3
  • 4
    How to you look at the resulting file? What operating system do you use? What filesystem? – Gereon Jul 02 '20 at 17:27
  • 4
    Perhaps you leave open the text editor which you use to view the file content, and *that* does not get updated. You might need to reload the file, and some editors warn you that the file content has changed. – Weather Vane Jul 02 '20 at 17:27
  • 1
    Test the return value: `if (fclose(fd)) perror("fclose");`. When there is no error, `fclose()` includes the equivalent of a `fflush()`: see [C11 7.21.5.1](http://port70.net/~nsz/c/c11/n1570.html#7.21.5.1). – pmg Jul 02 '20 at 17:28
  • 7
    In a case like this you should ideally check the result of **all four** i/o statements to verify they are supposed to have succeeded. – Weather Vane Jul 02 '20 at 17:30
  • 1
    What @WeatherVane noted about checking return values from I/O statements is critical. In the posted code, you can't know if ***ANY*** of your calls worked. Note also that the data in the `fprintf()` statements may not actually be written to the file until `fclose()` is called, and that `fclose()` **can fail** after all your other calls appeared to succeed because the data was buffered and not actually written until `fclose()` flushed it. If your disk gets full or other problems happen, all those previous "successful" `fwrite()` calls can be wiped out. – Andrew Henle Jul 02 '20 at 18:29
  • An addition to my comment about text editors: although some warn about changes to the file, I have never used a text editor that updates the *view* automatically, probably because it would be a bad policy. – Weather Vane Jul 02 '20 at 18:33
  • "10 extra strings appear" --> What are you using to read the file? – chux - Reinstate Monica Jul 02 '20 at 18:48
  • you need to check the return value of `fclose` also calling `fclose` is not ensuring that data is already on the disk. see my answer below. thank you. – Adam Jul 02 '20 at 19:32
  • When I take your code and run `tail -f test_file.txt` in another terminal, I get the extra lines every single time. I agree with others the problem is likely in what you use to view the file. – Zan Lynx Jul 02 '20 at 19:54

1 Answers1

3

Thanks to michael kerrisk we have all the answers we need in linux man pages. Will if you dig in the man pages and see the Notes section (listed below in my answer) you will understand this behavior.

int fclose(FILE *stream); - close a file stream pointed by stream.

DESCRIPTION

The fclose() function flushes the stream pointed to by stream (writing any buffered output data using fflush()) and closes the underlying file descriptor.

NOTES

Note that fclose() flushes only the user-space buffers provided by the C library. To ensure that the data is physically stored on disk the kernel buffers must be flushed too, for example, with sync() or fsync().

So its not enough to ensure writing and flushing your buffers, we need to flush the kernel buffers as well. how ? see below: make sure that you are reading my comments in the code as well!

#include <unistd.h>
#include <stdio.h>

int main(int argc, char const *argv[]) {
  FILE * fd;
  fd = fopen("test_file.txt", "a");
  fprintf(fd, "String 1\n");
  fprintf(fd, "String 2\n");
  
  if(fclose(fd) != 0){
/*********************************************************************
fclose failed and errno will be set to endicate the error. be aware 
that we shall 
not call fsync or any other stream manipulation on fd in this case 
because we will get undefined behavior !!!
**********************************************************************/
      ... do work but no more work on this stream (fd)...
  }

  /* ~~ be aware that this is a blocking call ! (see details below) ~~*/
  if(fsync(fd) == -1){
      /*fsync fails*/
      ....
  } 
  return 0;
}

fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) so that all changed information can be retrieved even if the system crashes or is rebooted. This includes writing through or flushing a disk cache if present. The call blocks until the device reports that the transfer has completed

One more important thing, always prefer calling fsync(fd) rather than calling void sync(void); ! why ?

because sync() causes all pending modifications to filesystem metadata and cached file data to be written to the underlying filesystems. usualy we dont want this kind of behavior (and this trigger the kernel to do extra work that is not essential! so dont call it.

this page and answer is not large enough to include all the details and special cases and all error codes! please refer to man pages and to the links belwo:

fclose

fsync

Adam
  • 2,820
  • 1
  • 13
  • 33
  • @piet_lu - i would be glad if you upvote and accept my answer if you find it useful (push the up arrow and V near by the answer start. thanks). – Adam Jul 02 '20 at 20:05
  • fsync takes a file descriptor, not a FILE pointer. Also, on modern-ish copy-on-write filesystems, fsync ends up flushing all the way to the root, since the metadata updates get caught in the storm. For that reason, it is effectively a no-op on zfs, hsfs, and a good many others. – mevets Jul 02 '20 at 21:24