How to hidden or lock file until task executed fully

Question

I have Perl script which will run for some time. It created file and write some data into it. I also have another Cron job which happens at 5.00 PM everyday.

My problem comes when my 1st script runs on 5.00 PM. Even when it is running(means that file don't have all details),it picks file from specified path.

How to prevent this?

I couldn't use flock in my script. I need another way to do this.

This might have an answer - similar question from not so long ago. http://stackoverflow.com/questions/40021131/detecting-that-files-are-being-copied-in-a-folder/40021570#40021570 — yoniyes, Oct 18 '16 at 12:54
If you can't use flock (for whatever reasons) you could use on of the numerous PID-Packages from CPAN, e.g. File::PID. Take a look @ http://stackoverflow.com/a/2232997/2678168 — Georg Mavridis, Oct 18 '16 at 15:03
Re "*I couldn't use flock in my script.*", Why not? It does exactly what you asked. If you don't tell us what your needs are, how do you expect us to fulfill them? — ikegami, Oct 18 '16 at 15:51
I appreciate this has been marked as a duplicate however solutions are supposed to be provided as answers not added into the question. — Bugs, Jun 23 '17 at 07:06

score 3 · Answer 1 · answered Oct 18 '16 at 12:53

3

I'd suggest take a look at flock - it lets you set an advisory lock on a file. But provided both your script and the cron job request a lock, then one will block the other until the lock is released (when the files is closed).

answered Oct 18 '16 at 12:53

Sobrique

52,974
7
60
101

Apart from flock() is there any other way in which I can achieve this? – Azarudeen Oct 18 '16 at 13:24
2

You specifically ask about how to lock a file. `flock` is how you do that. Alternatives... well, there's many, but "it depends". – Sobrique Oct 18 '16 at 15:02
1

@Azarudeen: "Apart from flock() is there any other way in which I can achieve this?" I wrote three alternatives in [my answer](http://stackoverflow.com/a/40109125/7231). We might be able to help more if you explained why you can't use `flock()`. – Dave Cross Oct 18 '16 at 15:05
I tried to use lock. But even when I use flock I can move that file into other location. Is this the excepted behavior of flock() – Azarudeen Oct 25 '16 at 13:24
Yes. All unix locking semantics - `flock` included - are advisory. Both processes need to check for locks. – Sobrique Oct 25 '16 at 13:33
I get error as could not lock file. – Azarudeen Oct 25 '16 at 14:03

score 2 · Accepted Answer · answered Oct 18 '16 at 13:05

2

There are several ways to do this.

Have the process that writes the file write it as a different name and only rename it to the expected name once the data has all been written.
Have the process that writes the file write another marker file once all the data has been written. Have your process which reads the file check for the existence of the marker file before reading the data file.
Use the standard Unix file locking feature. Perl has the flock() function which implements this.

answered Oct 18 '16 at 13:05

Dave Cross

68,119
3
51
97

1.Even I write by different name it will take that file(Cron script will take whichever file is in particular folder) – Azarudeen Oct 18 '16 at 13:23
3.I couldn't use lock file feature here. – Azarudeen Oct 18 '16 at 13:23

score 0 · Answer 3 · answered Oct 18 '16 at 13:35

One way to do this would be for the non-cron script to remove the production file when it first starts, write to a temporary file, then when it's done its work, copy the temp file to the prod file location. In the example code, the temp file will be removed by default when the script finishes.

Then, in the script that attempts to read the file at 5pm, have it loop and sleep until the prod file exists:

The script that writes the file:

use strict;
use warnings;

use File::Copy;
use File::Temp qw(tempfile);

my $prod_file = 'prod_dir/prod.txt';
unlink $prod_file or die $! if -e $prod_file;

my $tfh = File::Temp->new(UNLINK => 1);

while (...){
    ...;
    print $tfh "whatever\n";
}

copy $tfh->filename, $prod_file or die $!;

The script that reads the file at 1700 hrs:

use strict;
use warnings;

my $file = 'prod_dir/prod.txt';

while (! -e $file){
    sleep 10;
}

open my $fh, '<', $file or die $!;

while (<$fh>){
    ...; # do stuff
}

How to hidden or lock file until task executed fully

3 Answers3