9

In my production environment we have had what we believe to be a corrupt storable hash, created by Storable.pm. I am unable to replicate the behaviour in Dev, which has made it hard to diagnose exactly.

The code has been working for a long time, and the change that made it break was deleting from the hash. Up until recently, the hash either stayed the same size, or grew.

The file is opened in readwrite, and then store_fd writes to the file. As the hash is now (sometimes) smaller, it will write say 1000bytes to this 2000byte file. The tail 1000 bytes are old, garbage data. In my test cases, when I retrieve the hash, the garbage data is ignored, as expected.

open( $sf, "+< $self->{mod_state_filename}" );
flock( $sf, LOCK_EX );
$self->{mod_state} = fd_retrieve($sf);
delete ($self->{mod_state}{"somekey"});
seek( $sf, 0, 0 );
store_fd( $self->{mod_state}, $sf );
flock( $sf, LOCK_UN )
close($sf);

My questions:

  1. Should this work, or is it imperative that I truncate the file?
  2. Does the stored hash use some kind of file terminator character? If so, what is it?
  3. The above code, deleting and adding and deleting and adding, works perfectly in my test case. Can you suggest any test case sequence that might cause it to fail, due to the non-truncated file? (I know this is a really vague question, so feel free to ignore it).
Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
Brock
  • 160
  • 7
  • 2
    Why aren't you using storable's built in store and retrieve functions. Storable also supports locking http://search.cpan.org/~ams/Storable-2.24/Storable.pm#ADVISORY_LOCKING – singingfish Nov 24 '10 at 03:45
  • I know it does, but I didn't write the code. It was this way (except for the delete) years before I inherited it. I'd consider change the code to use Storable::locking, but I have no idea if it will (and suspect it probably won't) fix my problem. (I could try, but the problem only appears in production, so I need to be sure of the fix. I can't trial-and-error). I don't think my problem is due to locking. – Brock Nov 24 '10 at 04:21

2 Answers2

1

I don't know how well Storable deals with trailing garbage, but surely it can't hurt to add

truncate $sf, tell($sf);

after the call to store_fd, eliminating all doubt about whether it can deal with it now and in the future.

Peter S. Housel
  • 2,651
  • 1
  • 22
  • 25
  • Peter, don't call me Shirley. I agree, but I wanted to confirm it in test conditions first, hence the question. – Brock Dec 03 '10 at 04:16
0

Sorry, I thought I updated this.

I asked perl5-porters, and got answers to the question.

I've not implemented the fix, as I can't replicate under test, so don't want to push to my production env, my work-around is safer for the moment.

Truncating is definitely a good idea, based on the answers from perl5-porters.

I didn't know someone had (or could!) put a bounty on my question.

Brock
  • 160
  • 7