Stack Overflow
Stack Overflow

Questions tagged [snappy]

Snappy is a compression algorithm for byte streams and a library implementing this algorithm. The standard distribution includes bindings for C and C++; there are third-party bindings for many other languages.

Snappy does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses at about 500 MB/sec or more.

Snappy is widely used inside Google, in everything from BigTable and MapReduce to Google's internal systems.

366 questions
4
votes
3 answers

Set parquet snappy output file size is hive?

I'm trying to split parquet/snappy files created by hive INSERT OVERWRITE TABLE... on dfs.block.size boundary as impala issues a warning when a file in a partition is larger then block size. impala logs the following WARNINGS: Parquet files should…
Hatim Diab
  • 41
  • 1
  • 1
  • 4
4
votes
1 answer

Hive ORC compression

I run following code in hive v0.12.0 and I expect to get three tables compressed using different methods and therefore size and content of the files should be different. --- Create table and compress it with ZLIB create table zzz_test_szlib stored…
Andrey Dmitriev
  • 528
  • 2
  • 9
  • 27
4
votes
1 answer

snappy wkhtmltopdf wrapper send generated html file to browser

I googled like a mole, but can´t find the right way to go. I´m creating a PDF with the WKHTMLTOPDF Wrapper Snappy. How can I send the pdf generated with the method generateFromHtml straight to the Browser? Thats what I´m trying to…
Anatol
  • 1,923
  • 6
  • 26
  • 55
4
votes
0 answers

ElasticSearch: xerial.snappy error FAILED_TO_LOAD_NATIVE_LIBRARY

I'm trying running ElasticSearch client and getting xerial.snappy error FAILED_TO_LOAD_NATIVE_LIBRARY. I'm using elastic search v. 0.20.5: org.elasticsearch elasticsearch …
user1697575
  • 2,830
  • 1
  • 24
  • 37
4
votes
1 answer

Cassandra compression codebase

I want to know how many bytes are exactly stored on disk when I insert a new column in a Column Family of Cassandra. My main problem is that I need to know this information when columns are compressed with Snappy, I know the calculation of raw bytes…
Amanda
  • 941
  • 2
  • 12
  • 28
4
votes
1 answer

Snappy or LZO for logs then consumed by hadoop

I have a high volume service. I log events. Every few minutes, I zip the logs using gzip and rotate them to S3. From there, we process the logs using Amazon's Hadoop -- elastic mapreduce -- via Hive. Right now on the servers, we get a CPU spike…
John Hinnegan
  • 5,864
  • 2
  • 48
  • 64
3
votes
0 answers

snapd randomly takes up all the available memory

I'm using snaps in ubuntu 20.04. Every so often, even several times a day all of a sudden my computer freezes as the memory usage foes to 100% with all available memory taken by snapd. I tried killing the snapd process but it wouldn't make any…
marcin_j
  • 414
  • 4
  • 18
3
votes
1 answer

how does sortWithinPartitions sort?

After applying sortWithinPartitions to a df and writing the output to a table I'm getting a result I'm not sure how to interpret. df .select($"type", $"id", $"time") .sortWithinPartitions($"type", $"id", $"time") result file looks somewhat like 1 a…
gjin
  • 860
  • 1
  • 14
  • 28
3
votes
3 answers

How to solve "The process has been signaled with signal 11" - Snappy Library?

I am using a Snappy PDF library in Laravel 5.7. Locally it works everything fine, but in my dev environment I get this error: The process has been signaled with signal "11". Till now I was not able to find a solution that would help me. This is my…
Felicity
  • 63
  • 1
  • 1
  • 8
3
votes
0 answers

Exit with code 1 due to network error: ConnectionRefusedError when generating PDF file from symfony running in Docker

I am trying to generate a pdf from a twig page by using snappy bundle with wkhtmltopdf,: $pageUrl = $command->getRouter()->generate('testDevis',array('idDevis'=>$command->getIdDevis()),UrlGeneratorInterface::ABSOLUTE_URL); return new…
Suhairi Subhi
  • 31
  • 2
3
votes
1 answer

Laravel-Snappy throwing error 500 without further information

I want to create PDF files using barryvdh\laravel-snappy. I followed all the steps on the github page, but whenever I use functions like $pdf->output(); $pdf->download("test.pdf"); I get an Server Error. I tried to install wkthmltopdf on windows…
Commander
  • 264
  • 2
  • 12
3
votes
0 answers

How to read Spark log files? .Iz4 or .snappy

I want to read some logs but I cannot. So far, I've tried: hadoop fs -text But the only thing I get is: INFO compress.CodecPool: Got brand-new decompressor [.lz4] (same for .snappy) val rawRdd =…
mjbsgll
  • 722
  • 9
  • 24
3
votes
2 answers

Javascript - Read parquet data (with snappy compression) from AWS s3 bucket

In nodeJS, I am trying to read a parquet file (compression='snappy') but not successful. I used https://github.com/ironSource/parquetjs npm module to open local file and read it but reader.cursor() throws cryptic error 'not yet implemented'. It does…
Test Test
  • 145
  • 1
  • 14
3
votes
1 answer

Decrypting Hadoop Snappy File

So I'm having some issues decrypting a snappy file from HDFS. If I use hadoop fs -text I am able to uncompress and output the file just file. However if I use hadoop fs -copyToLocal and try to uncompress the file with python-snappy I get…
anoneironaut
  • 1,778
  • 16
  • 28
3
votes
4 answers

Displaying an image in pdf file generated by snappy

I want to add an image to my pdf generated by snappy ,but nothing appear In my twig : I am persisting the name of the file in my data base, survey.event.image contain the name of the image
Julie
  • 563
  • 6
  • 10
  • 23
1 2 3
…
24 25