Questions tagged [compression]

The name given to the process of encoding data such that it uses lesser number of bits as compared to the original representation.

There are a number of techniques to compress information, with most algorithms working to compress image, speech, music or video. Encoding the same information in lesser bits results in saving some crucial resource such as transmission bandwidth or hard-disk space. Compression algorithms also take advantage of human perception to further reduce the size of information. By encoding the same piece of information with fewer bits in places that the human brain is not accustomed to perceiving as well as other parts, substantial gains in compression ratios can be achieved.

9175 questions
27
votes
6 answers

Compress numpy arrays efficiently

I tried various methods to do data compression when saving to disk some numpy arrays. These 1D arrays contain sampled data at a certain sampling rate (can be sound recorded with a microphone, or any other measurment with any sensor) : the data is…
Basj
  • 41,386
  • 99
  • 383
  • 673
27
votes
4 answers

how to unpack resources.pak from google chrome?

There are bunch of interesting files accessible via chrome://resources/* using google chrome. On linux That the content is in /opt/google/chrome/resources.pak. I know I can get the whole sources from http://chromium.googlecode.com/svn/trunk/ but I…
Michał Šrajer
  • 30,364
  • 7
  • 62
  • 85
26
votes
8 answers

zip/compress a folder full of files on android

I need to zip up a "project" folder to allow users to share projects via email. I found a class for zipping up multiple files into one zip, but I need to keep the folder structure in my zip. Is there any way to achieve this on android? Thanks in…
Mark
  • 700
  • 2
  • 7
  • 17
26
votes
3 answers

Fiddler doesn't decompress gzip responses

I use Fiddler to debug my application. Whenever the response is compressed by server, instead of decompressed response, Fiddler shows unreadable binary data: /* Response to my request (POST) */ HTTP/1.1 200 OK Server: xyz.com Date: Tue, 07 Jun 2011…
Xaqron
  • 29,931
  • 42
  • 140
  • 205
26
votes
5 answers

Hadoop, how to compress mapper output but not the reducer output

I have a map-reduce java program in which I try to only compress the mapper output but not the reducer output. I thought that this would be possible by setting the following properties in the Configuration instance as listed below. However, when I…
Marcin
  • 684
  • 2
  • 9
  • 19
26
votes
8 answers

iPhone Unzip code

Really stuck on trying to write code to unzip a file or directory on the iPhone. Below is some sample code that I'm using to try and unzip a simple text file. It unzips the file but its corrupt. (void)loadView { NSString *DOCUMENTS_FOLDER =…
TonyNeallon
  • 6,607
  • 12
  • 42
  • 49
26
votes
2 answers

How to compress a Byte array without stream or system io

I'm trying to encode an image into a byte array and send it to a server. the encoding and sending parts wok fine but my problem is that the byte array is too large and takes too long to send so I thought compressing it would make it go faster. but…
Henjin
  • 528
  • 2
  • 9
  • 19
26
votes
3 answers

Is gzip compression useful for mobile devices?

I'm wondering if anyone has a clue on whether the gzip compression is as much useful on mobile devices than it is on a desktop computer. Will the phone use more battery? Or will it save some because of the bandwidth saving? Will the page page load…
Savageman
  • 9,257
  • 6
  • 40
  • 50
26
votes
1 answer

Compressing content with PHP ob_start() vs Apache Deflate/Gzip?

Most sites want to compress their content to save on bandwidth. However, When it comes to apache servers running PHP there are two ways to do it - with PHP or with apache. So which one is faster or easier on your server? For example, in PHP I run…
Xeoncross
  • 55,620
  • 80
  • 262
  • 364
26
votes
1 answer

Is Vary: Accept-Encoding overkill?

After reading about how gzip compression works it got me thinking. If the Origin and Proxy server (CDN) both support gzip is adding a Vary: Accept-Encoding header necessary?
jdorfman
  • 781
  • 2
  • 10
  • 21
26
votes
3 answers

Compress and decompress a Stream with Compression.DeflateStream

I am trying to compress and decompress a Stream using Compression.DeflateStream. Compressing seems to work correctly since the code below compresses my Stream to a 110 bytes long array. However, reading the decompressed Stream results in an empty…
greijner
  • 299
  • 1
  • 3
  • 10
25
votes
4 answers

Python/Pandas create zip file from csv

Is anyone can provide example how to create zip file from csv file using Python/Pandas package? Thank you
Felix
  • 1,539
  • 8
  • 20
  • 35
25
votes
3 answers

Spark: writing DataFrame as compressed JSON

Apache Spark's DataFrameReader.json() can handle gzipped JSONlines files automatically but there doesn't seem to be a way to get DataFrameWriter.json() to write compressed JSONlines files. The extra network I/O is very expensive in the cloud. Is…
Sim
  • 13,147
  • 9
  • 66
  • 95
25
votes
5 answers

How to create minimum size (empty) zip file, which has 22B?

I´m trying to solve some security problem with file, whose MD5 hash is 76cdb2bad9582d23c1f6f4d868218d6c. I don't have that file, but based on internet MD5 searches, I think, it is minimum size zip file. I found one example and its content is exactly…
mirekbarton
  • 391
  • 1
  • 3
  • 6
25
votes
2 answers

Uncompress BZIP2 archive

I can uncompress zip, gzip, and rar files, but I also need to uncompress bzip2 files as well as unarchive them (.tar). I haven't come across a good library to use. I am using Java along with Maven so ideally, I'd like to include it as a dependency…
Walter White