In C++, I use the GzipOutputStream
to compress the data, which is a big string. At first, I have to convert the string to a proto message, like this:
syntax = "proto3";
package Serverlog_Compress;
message ServerlogMessage {
string data = 1;
}
Then, in the code, I call the GzipOutputStream
to compress the data, like:
google::protobuf::io::GzipOutputStream::Options options_;
options_.format = google::protobuf::io::GzipOutputStream::GZIP;
options_.compression_level = 9;
google::protobuf::io::FileOutputStream file_stream(fd);
google::protobuf::io::GzipOutputStream gzip_stream(&file_stream, options_);
auto *message = butil::get_object<Serverlog_Compress::ServerlogMessage>();
message->set_data(data);
int ret = 0;
{
std::lock_guard<std::mutex> lock(file_mutex_);
ret = message->SerializeToZeroCopyStream(&gzip_stream);
}
if (!ret) {
LOG(ERROR) << "Failed to write the message, the ret is " << ret;
message->Clear();
butil::return_object(message);
return -1;
}
message->Clear();
butil::return_object(message);
After I finish the compression, I got the compressed data file, but when I use the zcat
to get the file data, I got this error:
gzip: 20200301_12: invalid compressed data--crc error
gzip: 20200301_12: invalid compressed data--length error
20200301_12
is the file name.