How do I give parallel write access to a file without collisions?

Question

I have some child processes which should write some logs into a common file. I am wondering if this code work so that the processes will write into the common file without collisions:

sub appendLogs {
    open FILE, "+>>", $DMP or die "$!";
    flock FILE, LOCK_EX or die "$!";
    print FILE "xyz\n";
    close FILE;
}

If not, could you give me any hints, how I could fix or improve it?

score 4 · Answer 1 · edited May 23 '17 at 12:12

4

For logging purpose, I would use Log4perl instead of reinventing the wheel. It has a support for what you are looking.

edited May 23 '17 at 12:12

Community

1
1

answered Jun 15 '12 at 13:00

Bala

4,427
6
26
29

thank you very much for feedback!! well.. altough i have written, i need an algorithm for logging, i... don't really mean a logging... i must save some part of xml file... – user897237 Jun 15 '12 at 13:21

score 3 · Accepted Answer · answered Jun 15 '12 at 12:51

3

Yes, as long as every process that tries to write to file uses flock, they will go without collisions.

answered Jun 15 '12 at 12:51

Oleg V. Volkov

21,719
4
44
68

But what if I need to write to file without an option to die if I cannot obtain successful `flock`? Your solution is not good enough for critical apps. – Ωmega Oct 21 '19 at 14:36
@Ωmega you don't write to a single point of failure in "critical apps". And this question is about logs. – Oleg V. Volkov Nov 24 '19 at 09:48

mob · Answer 3 · 2012-06-15T20:25:38.477

0

If you would like your code to be portable, you should seek to the end of the file after you lock the filehandle but before you write to it. See the "mailbox appender" example in perldoc -f flock, which is similar to what you are doing.

sub appendLogs {
    open FILE, "+>>", $DMP or die "$!";
    flock FILE, LOCK_EX or die "$!";
    seek FILE, 0, 2;       # <--- after lock, move cursor to end of file
    print FILE "xyz\n";
    close FILE;
}

The seek may be necessary because another process could append the file (and move the position of the end of the file) after you open the file handle but before you acquire the lock.

edited Jun 15 '12 at 20:25

answered Jun 15 '12 at 15:06

mob

117,087
18
149
283

On linux (or POSIX in general?), the seek is unnecessary. "If the file was open(2)ed with O_APPEND, the file offset is first set to the end of the file before writing. The adjustment of the file offset and the write operation are performed as an atomic step." – ikegami Jun 15 '12 at 17:58
@ikegami - fair enough. I rewrote my first sentence. – mob Jun 15 '12 at 20:25
What if dying after unsuccessful `flock` call is not an option? – Ωmega Oct 21 '19 at 14:50

How do I give parallel write access to a file without collisions?

3 Answers3