Read zst compressed pcap with libpcap

Question

I would like to decode zst and gz pcap with libpcap but I am not able to find any example doing so. Of course I don't want to have a temporary decompressed pcap file. Could you guys point me to the right methods ?

Thanks

score 2 · Answer 1 · answered Jun 07 '22 at 22:44

On linux, you can use libz to open gzip files, and fopencookie() to remap the read/write/seek/close functions to the libz equivalents.

//-------------------------------------------------------------------------
ssize_t gzip_cookie_read( void * cookie, char * buf, std::size_t size )
{ return ::gzread( reinterpret_cast<gzFile>(cookie)
                 , buf
                 , size
                 ) ;
}
ssize_t gzip_cookie_write( void * cookie, const char * buf, std::size_t size )
{ return ::gzwrite( reinterpret_cast<gzFile>(cookie)
                  , buf
                  , size
                  ) ;
}
int32_t gzip_cookie_seek( void * cookie, off64_t * off, int32_t whence )
{
  z_off_t rv = ::gzseek( reinterpret_cast<gzFile>(cookie)
                       , *off
                       , whence
                       ) ;
  if( rv >= 0 )
  { *off = rv ;
    return 0 ;
  }
  return -1 ;
}
int32_t gzip_cookie_close( void * cookie )
{ return ::gzclose( reinterpret_cast<gzFile>(cookie) ) ;
}

//------------------------------------------------------------------
//
cookie_io_functions_t
gzip_cookie_functions =
{ .read  = gzip_cookie_read
, .write = gzip_cookie_write
, .seek  = gzip_cookie_seek
, .close = gzip_cookie_close
} ;

... Then

  char err_buf[ PCAP_ERRBUF_SIZE + 1] ;
  std::memset( err_buf, 0, sizeof(err_buf) ) ;
  gzFile gzf = ::gzopen( args[1].c_str(), "r" ) ;
  if( gzf == Z_NULL )
  { std::cout
      << "Error: Failed to open input file (via gzopen())" << std::endl
      << "(" << err_buf << ")" << std::endl
      << std::endl ;
    return EXIT_FAILURE;
  }

  // Convert gzFile handle to FILE pointer w/ same underlying stream.
  FILE * pcfp = ::fopencookie( gzf, "r", gzip_cookie_functions ) ;
  if( pcfp == nullptr )
  { std::cout
      << "Error: Failed to redirect file i/o functions through zlib via funopen()" << std::endl
      << "(" << err_buf << ")" << std::endl
      << std::endl ;
    return EXIT_FAILURE;
  }

  ::pcap_t * pch = ::pcap_fopen_offline_with_tstamp_precision
                     ( pcfp
                     , PCAP_TSTAMP_PRECISION_NANO
                     , err_buf
                     ) ;
  if( pch == nullptr )
  { std::cout
      << "Error: Failed to open input file" << std::endl
      << "(" << err_buf << ")" << std::endl
      << std::endl ;
    return EXIT_FAILURE;
  }

score 1 · Accepted Answer · answered Oct 06 '21 at 07:43

1

Libpcap doesn't currently support reading compressed files.

If you don't want to have a temporary decompressed pcap file, the only way to do so would be to have your program create a pipe, run another program that reads the compressed file and writes the decompressed file data to the standard output with its standard output being the write side of the pipe, make a standard I/O stream from the read side of the pipe (using, for example, fdopen() on a Unix-like system), and then use pcap_fopen_offline() to open that standard I/O stream as a capture file.

That way, the other program will do the decompression, but will write it to a pipe rather than to a file, and your program will read the decompressed data from the pipe.

answered Oct 06 '21 at 07:43

user16139739

862
3
5

That's what I was afraid of. Thank you for your help. – kehezov Oct 06 '21 at 13:05
Or you could write a program that opens the standard input as a capture file and reads from it, and pipe the decompressing command to that program. If you pass `"-"` as the argument to `pcap_open_offline()`, it will open the standard input. – user16139739 Oct 06 '21 at 21:54
Yeah but I would like to handle everything in a single application. – kehezov Oct 06 '21 at 22:16
Then you will need to have multiple threads in your application, with one thread decompressing the file to a pipe and another thread opening the pipe as a `pcap_t` and reading form it - and you'll need to incorporate decompression code into your application. – user16139739 Oct 07 '21 at 16:46
Yeah I am completely fine with this. I'll dump a code sample here when I will be done. – kehezov Oct 08 '21 at 14:46

Read zst compressed pcap with libpcap

2 Answers2