atomic append on a file descriptor, but at what offset?

Question

in unistd.h

using open() with the O_APPEND flag gives atomic writes always to the end of the file...

this is great and all, but what if i need to know the offset at which it atomically appended to the file...?

i realize O_APPEND is often used for log files, but I'd actually like to know at what offset in the file it atomically appended.

I don't see any obvious way to do this..? Does anyone know?

Thanks

The append operation is not atomic, unless you write unbuffered data, one sector at the time. — ROAR, Nov 17 '10 at 17:28
If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation. http://www.opengroup.org/onlinepubs/009695399/functions/pwrite.html — Michael Xu, Nov 17 '10 at 19:02

score 0 · Accepted Answer · answered Nov 17 '10 at 17:18

0

To get the current position in a file descriptor, use lseek() with offset 0 and whence SEEK_CUR.

int fd = open(...);
if (fd) {
    off_t positionWhereAppendingBegins = lseek(fd, 0, SEEK_CUR);
    write(...);
    close(fd);
}

Note that this will not give you reliable results if the descriptor was opened some other way, i.e. via socket().

answered Nov 17 '10 at 17:18

Jonathan Grynspan

43,286
8
74
104

3

but this would be inaccurate, because on appends the offset is updated atomically inside of the write call, to the end of the file. – Michael Xu Nov 17 '10 at 17:23
It doesn't go back and modify `positionWhereAppendingBegins`. That value remains the same until it goes out of scope. – Jonathan Grynspan Nov 18 '10 at 01:00
2

The point is that an intervening `write()` by another process between the `lseek()` and the `write()` will mean that the write happens at a different offset to the one returned by `lseek()`. – caf Nov 18 '10 at 03:34
In my example, the file descriptor has *just* been opened. Nobody else knows about it yet. In a well-formed program, nobody else has had a chance to call `write()` with it. – Jonathan Grynspan Nov 18 '10 at 03:54

David Harris · Answer 2 · 2010-11-17T22:33:58.987

The file is written to at the file offset as obtained by the process when the file was opened. If another process writes to the file between the open and the write, then contents of the file are indeterminate.

The correct method of handling multiple process writing to a single file is for all processes to open the file with the O_APPEND flag, obtain an exclusive lock and once the lock is obtained, seek to the end of the file before writing to the file, and finally close the file to release the lock.

If you want to keep the file open between writes, initiate the process by opening the file with the O_APPEND flag. The writing loop in this case is obtain the exclusive lock, seek to the end of the file, write to the file and release the lock.

If you really need the file position, lseek will return the file offset of the callers file descriptor at the time of the call.

atomic append on a file descriptor, but at what offset?

2 Answers2