Why does underscore in path name make Perl script hang?

Question

I'm troubleshooting a Perl script that unexpectedly hanging when it never hung before. I don't know Perl. I finally traced the problem to a file path string. This code works:

$eng_morph = "~/datafile.en.db";
tie %eng_morph, "DB_File", $eng_morph, O_CREAT|O_RDWR, 0664|| die "Cannot open dbmfile $eng_morph";

When I change the file name to include an underscore, the second line hangs forever:

$eng_morph = "~/datafile.en_us.db";
tie %eng_morph, "DB_File", $eng_morph, O_CREAT|O_RDWR, 0664|| die "Cannot open dbmfile $eng_morph";

Is there something wrong with the syntax? Is there any way to allow the underscores?

I'm using Ubuntu 14.04. Here's the uname output:

Linux asus-notebook 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Also, I'm also using the default Perl in the distro. Its version output is:

This is perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-linux-gnu-thread-multi (with 41 registered patches, see perl -V for more detail)

your code has an unrelated bug: `||` should be `or`. `||` is high-precedence, so instead of dying when tie fails, it is dying when 0664 is false (which is never) — ysth, Jan 11 '15 at 07:09
@ysth - this might not be "unrelated". Maybe this error is masking the real problem and the "which is never" dying part is taking over. I'm not familiar with Perl and don't understand the syntax of your correction. Can you please put the complete correction in an answer so I can try it? — tahoar, Jan 12 '15 at 07:42

score 1 · Answer 1 · answered Jan 11 '15 at 06:50

1

There is almost certainly something wrong with that specific DB file, and not the file name.

Either the DB file has gotten corrupted in some fashion, or an existing process on the server has the file open and has locked it.

See if any other process has the file open (using lsof), and check for files named similarly to the database but beginning with "." in the same directory. (i.e. do ls -a ~ | grep -i en_us)

answered Jan 11 '15 at 06:50

Daniel Martin

23,083
6
50
70

It's impossible to be a corrupted DB file because the output folder is created specifically for this output. The Perl code then creates the DB file in the empty output folder. However, I agree it's not the underscore. The lock-up problem finally occurred with a filename without the underscore. – tahoar Jan 12 '15 at 07:40

score 1 · Answer 2 · answered Jan 12 '15 at 09:01

tie %eng_morph, "DB_File", $eng_morph, O_CREAT|O_RDWR, 0664|| die "Cannot open dbmfile $eng_morph";

needs to be:

tie %eng_morph, "DB_File", $eng_morph, O_CREAT|O_RDWR, 0664 or die "Cannot open dbmfile $eng_morph";

or

tie( %eng_morph, "DB_File", $eng_morph, O_CREAT|O_RDWR, 0664 ) || die "Cannot open dbmfile $eng_morph";

because || is a high precedence operator. Otherwise it is interpreted as:

tie %eng_morph, "DB_File", $eng_morph, O_CREAT|O_RDWR, ( 0664 || die "Cannot open dbmfile $eng_morph" );

|| is designed to naturally be used in an expression that returns a result; or is designed to be used for flow control between what are essentially different expressions (though the two differ only in precedence).

Because of this error, when the tie fails, you don't die and the code continues running, but leaving %eng_morph as a normal, untied hash.

Thanks, ysth. I've updated the code, but it didn't solve the hang or reveal an underlying error. I did, however find the problem and wrote my own answer. — tahoar, Jan 14 '15 at 02:22

score 0 · Accepted Answer · answered Jan 14 '15 at 02:37

This problem was intermittent. It turned out to be a threading problem -- as many intermittent problems are. The clue was the presence of an odd file in the output folder: "~/_db_datafile.en.db" ... or any defined output file name prefixed with "~/_db_". This output file ranged from 0 to 14K bytes when the desired output file should have been 2 or 3 megabytes. It looked like the Perl tie() function was interrupted while creating the DB file.

The code in question was being called from a Python script that used non-blocking threads with subprocess.Popen() piping. The Python script spawned the Perl script and continued its flow. While the Perl script was still creating the DB file, Python's non-blocked thread started piping data. This stopped the Perl script's file creation and caused a lockup.

The underscore in the file name was only the first intermittent instance of this error, and ultimately irrelevant to the problem.

Resolution was to simply create a while not os.path.exists(eng_morph): loop in the Python code to pause until the proper DB file exists.

Why does underscore in path name make Perl script hang?

3 Answers3