0

Application Scenario:

I receive many network packets, and group them by tcp stream. I can make sure that the packets that from one specific. The first received packet is the head of gzip content(0x1f 0x8b 0x08...), so I initialize the zlib_steam and then can decompress this packet successfully, but I cannot make sure the following received packet is Arrive in order(they may be out-of-order or lost), When the received packet is not the next packet of pre-packet, zlib_steam mostly failed to decompress the packet.

So I have the question: If you give me a partial compressed gzip data(for example one HTTP packet with gzip content-encoding but it's just a part of the complete gzip content), can zlib decompress it successfully?

Pranav Singh
  • 17,079
  • 30
  • 77
  • 104
zs eric
  • 11
  • 2
  • 1
    No, you need the packets in order. – pmqs Mar 30 '20 at 13:48
  • Is this the same question as from four days ago? [zlib inflate error : Z_DATA_ERROR while the received packets is out-of-order or lost](https://stackoverflow.com/questions/60860604/)? – U880D Mar 30 '20 at 15:52
  • @U880D They seem to be quite similar. The other question appears to be more detailed, too. – AMC Mar 30 '20 at 18:24
  • @U880D yes, That question was also raised by me, thank you for your attention. For each packet that arrives out of order, there is no need to fully decompress it, as long as the data can be partially decompressed. If you have a solution, please tell me, thanks a lot. – zs eric Mar 31 '20 at 01:58
  • @AMC thanks for your attention. Since I haven't received a reply in the past few days, I am worried that the question will be too long and reader will get bored, so I have selected some questions to ask. If you have a solution, please tell me, thanks a lot. – zs eric Mar 31 '20 at 02:01
  • @pmqs thanks for your answer. Since my scenario can only process a single data packet and then maintain it through streams. For each packet that arrives out of order, if I don't require all the decompression, is there any solution to make the data as decompressible as possible? – zs eric Mar 31 '20 at 02:04
  • Looking at your other post it appears you are processing real internet traffic, so you don't have any control of the type of compressed data you are receiving. Is that correct? If you cannot store state to ensure the packets are processed in order, you could try using the inflateSync function in zlib. That can skip invalid data until it finds a flush point. This though is dependent on the code doing the compression actually forcing a flush point. I suspect that will not be the case, but worth a look. – pmqs Mar 31 '20 at 09:41
  • @pmqs Thank you for your reply. yes, it's real internet traffic, and I can't make sure the packets are processed in order. I have tried inflateSync function, but it doesn't work. there's no change. no flush point has been found and inflateSync return Z_DATA_ERROR. If you have other solution, please let me know. Anyway, thank you very much for your reply! – zs eric Apr 01 '20 at 03:28

0 Answers0