Stack Overflow
Stack Overflow

Questions tagged [compression]

The name given to the process of encoding data such that it uses lesser number of bits as compared to the original representation.

There are a number of techniques to compress information, with most algorithms working to compress image, speech, music or video. Encoding the same information in lesser bits results in saving some crucial resource such as transmission bandwidth or hard-disk space. Compression algorithms also take advantage of human perception to further reduce the size of information. By encoding the same piece of information with fewer bits in places that the human brain is not accustomed to perceiving as well as other parts, substantial gains in compression ratios can be achieved.

9175 questions
43
votes
8 answers

Implementing in-memory compression for objects in Java

We have this use case where we would like to compress and store objects (in-memory) and decompress them as and when required. The data we want to compress is quite varied, from float vectors to strings to dates. Can someone suggest any good…
Kichu
  • 667
  • 1
  • 6
  • 9
43
votes
8 answers

tar: Error is not recoverable: exiting now

when I untar doctrine -rw-r--r-- 1 root root 660252 2010-10-16 23:06 Doctrine-1.2.0.tgz I always get this error messages root@X100e:/usr/local/lib/Doctrine/stable# tar -xvzf…
udo
  • 4,832
  • 4
  • 54
  • 82
43
votes
13 answers

Are there any downsides to using UPX to compress a Windows executable?

I've used UPX before to reduce the size of my Windows executables, but I must admit that I am naive to any negative side effects this could have. What's the downside to all of this packing/unpacking? Are there scenarios in which anyone would…
Mick
  • 13,248
  • 9
  • 69
  • 119
43
votes
5 answers

Data Compression Algorithms

I was wondering if anyone has a list of data compression algorithms. I know basically nothing about data compression and I was hoping to learn more about different algorithms and see which ones are the newest and have yet to be developed on a lot of…
Veridian
  • 3,531
  • 12
  • 46
  • 80
43
votes
2 answers

Does PostgreSQL support transparent compressing of tables (fragments)?

I'm going to store large amount of data (logs) in fragmented PostgreSQL tables (table per day). I would like to compress some of them to save some space on my discs, but I don't want to lose the ability to query them in the usual manner. Does…
Prikrutil
  • 2,610
  • 3
  • 18
  • 12
43
votes
12 answers

How to compare two tarball's content

I want to tell whether two tarball files contain identical files, in terms of file name and file content, not including meta-data like date, user, group. However, There are some restrictions: first, I have no control of whether the meta-data is…
myjpa
  • 433
  • 1
  • 4
  • 7
42
votes
2 answers

What is the best file compression of random binary data that you can achieve?

Specifically, what programs are out there and what has the highest compression ratio? I tried Googling it, but it seems experience would trump search results, so I ask.
ox_n
  • 657
  • 1
  • 7
  • 11
42
votes
5 answers

How to compress a buffer with zlib?

There is a usage example at the zlib website: http://www.zlib.net/zlib_how.html However in the example they are compressing a file. I would like to compress a binary data stored in a buffer in memory. I don't want to save the compressed buffer to…
Richard Knop
  • 81,041
  • 149
  • 392
  • 552
42
votes
9 answers

Is it better to encrypt a message and then compress it or the other way around? Which provides more security?

I have the assumption there is no added protection at all.
john
  • 435
  • 1
  • 4
  • 4
41
votes
4 answers

Choosing a video codec for screen recording

I'm doing a series of tutorials by recording the screen and my voice. Just as PNG works best for screenshots, while JPEG is better suited for photographs (although lossy), which video encoding & container formats are better for screen…
Dheeraj Vepakomma
  • 26,870
  • 17
  • 81
  • 104
41
votes
7 answers

How can I tail a zipped file without reading its entire contents?

I want to emulate the functionality of gzcat | tail -n. This would be helpful for times when there are huge files (of a few GB's or so). Can I tail the last few lines of such a file w/o reading it from the beginning? I doubt that this won't be…
baskin
  • 1,643
  • 1
  • 18
  • 29
40
votes
11 answers

How do I concatenate JavaScript files into one file?

I want to create a compiled JavaScript file for my website. For development I would prefer to keep the JavaScript in separate files and just as part of my automated scripts concatenate the files into one and run the compressor over it. My problem…
Phil Hannent
  • 12,047
  • 17
  • 71
  • 118
39
votes
6 answers

How to tell if a file is gzip compressed?

I have a Python program which is going to take text files as input. However, some of these files may be gzip compressed. Is there a cross-platform, usable from Python way to determine if a file is gzip compressed or not? Is the following…
Ryan Gabbard
  • 2,269
  • 2
  • 24
  • 37
39
votes
6 answers

How can I use the "compress/gzip" package to gzip a file?

I'm new to Go, and can't figure out how to use the compress/gzip package to my advantage. Basically, I just want to write something to a file, gzip it and read it directly from the zipped format through another script. I would really appreciate if…
pymd
  • 4,021
  • 6
  • 26
  • 27
39
votes
6 answers

High quality JPEG compression with c#

I am using C# and want to save images using JPEG format. However .NET reduces quality of the images and saves them with compression that is not enough. I want to save files with their original quality and size. I am using the following code but…
Baran
  • 1,388
  • 6
  • 16
  • 33
1 2 3
…
99 100