-1

edit: the file path i am giving is not /tmp/...

the file path is in the form of: Erfurt/phased_edited_all_chr.gz

i think the tmp directory is created by Slurm, but not sure about it. end edit.

I am working on a data analysis project, and have a script that use this command:

haps_file <- fread(HAPS, drop=c(2), colClasses = list(integer = 1:3, character = 4:5))

i run this script about 30~ times with the same HAPS input (but other changes to the script). HAPS is a gzip file of about 200 Mb

Most of the times it runs OK. Three of the times (33) it resulted with this error:

Error in fread(HAPS, drop = c(2), colClasses = list(integer = 1:3, character = 4:5)) : File is empty: /tmp/Rtmp998Hni/file9cde5383550b

The scripts are executed with slurm using a bash command.

Any idea what happens here? Each job works from a different core and it should work. Any idea how to solve the problem?

Thanks!

whitestorn
  • 1
  • 1

1 Answers1

0

Since the location is in /tmp, the OS might be cleaning it every so often -

https://serverfault.com/questions/377348/when-does-tmp-get-cleared/377349

Can you try moving the file location out of /tmp?

Alex Joseph
  • 4,719
  • 2
  • 21
  • 25
  • the file is located on the university servers. the path i enter there is a path to my folder on that server. i think the /tmp location is created for the job itself by the slurm clusters? i dont really know. the path itself is more like /vol/sci/.../phased_edited_all_chr.gz – whitestorn Nov 16 '20 at 08:02