5

I wrote a short category on NSData that does compression with libSnappy. It works like a charm during compression, however the decompression code results in SNAPPY_INVALID_INPUT. The interesting part is, despite the invalid Op-Code, snappy still fills my result's buffer with uncompressed data. Does anybody know why snappy is throwing this at me?

- (NSData*)dataBySnappyUncompression {
    NSMutableData *result = nil;
    if (self.bytes != NULL) {
        size_t uncompress_result = 0;
        snappy_status opCode = snappy_uncompressed_length(self.bytes, self.length, &uncompress_result);
        if (opCode == SNAPPY_OK) {
            result = [NSMutableData dataWithLength:uncompress_result];
            opCode = snappy_uncompress(self.bytes, self.length, [result mutableBytes], &uncompress_result);
            if (opCode == SNAPPY_OK) {
                [result setLength:uncompress_result];
                return result;
            }
        }
    }
    LEPLog(@"Failed snappy de-compress: tried to de-compress %lu bytes", self.length);
    NSAssert(nil, @"Failed Snappy de-compress");
    result = nil;
    return result;
}

I should note that the data that is being compressed/uncompressed is the result of an NSKeyedArchiver call.

Mike Abdullah
  • 14,933
  • 2
  • 50
  • 75
CodaFi
  • 43,043
  • 8
  • 107
  • 153
  • Which check of the return code is failing? – jscs Jan 20 '13 at 19:41
  • the inner call to `snappy_uncompress` – CodaFi Jan 20 '13 at 19:41
  • And the uncompressed data is _correct_ despite the indication of failure? – jscs Jan 20 '13 at 19:42
  • Yes, surprisingly. I even ran a test against the original data that was compressed (before it was compressed) and it was correct. – CodaFi Jan 20 '13 at 19:43
  • @CodaFi Note that providing a char or void pointer has nothing to do with the compression/the error. Are you sure it does indeed return an error code? –  Jan 20 '13 at 19:45
  • @H2CO3 Yes. LLDB was quite explicit about it, and a quick `p opCode` returns SNAPPY_INVALID_INPUT – CodaFi Jan 20 '13 at 19:46
  • @CodaFi That's strange, indeed. I'm checking the snappy source tree. –  Jan 20 '13 at 19:49
  • @H2CO3 You can find the C-bindings I've used [here](http://code.google.com/p/snappy/source/browse/trunk/snappy-c.cc?r=27). – CodaFi Jan 20 '13 at 19:50
  • @CodaFi I don't see anything particularly wrong... I guess this might need some more profound investigation... –  Jan 20 '13 at 20:01
  • @CodaFi: I have downloaded the snappy source code and compiled it together with your category uncompression method (and added a similar compression method). With 10.000 random data blocks of length 10.000 I could not reproduce the problem. – Martin R Jan 30 '13 at 20:26
  • is the value of self.length in the call to snappy_uncompressed_length correct/as-expected going in? what is the value of uncompressed_result after it is done? is it the proper length of the compressed data after all? – john.k.doe Feb 01 '13 at 05:36
  • @john.k.doe it's all completely valid, even the length calls. Even NSKeyedUnarchiver is happy with the outcome and produces valid objects every time. – CodaFi Feb 01 '13 at 05:38
  • 1
    interesting. i take it you don't have as much time on your hands as Martin R, to get their source and set bkpts on the locations where SNAPPY_INVALID_INPUT may be set and interpret the circumstances? with each comment, sounds more and more like snappy has some math bug w.r.t. it's compression vs uncompression as it relates specifically to your input. – john.k.doe Feb 01 '13 at 05:44
  • @john.k.doe my working theory is that it doesn't like blocks that aren't perfectly 8-bytes square on the end. All of my data appears to have the ends be only 4 bytes long. – CodaFi Feb 01 '13 at 06:04
  • @CodaFi: I tried with random data of various length, and cannot reproduce the issue. Is there any chance that you can create and publish a small project showing the problem? – Martin R Feb 06 '13 at 17:45

0 Answers0