Questions tagged [compression]

The name given to the process of encoding data such that it uses lesser number of bits as compared to the original representation.

There are a number of techniques to compress information, with most algorithms working to compress image, speech, music or video. Encoding the same information in lesser bits results in saving some crucial resource such as transmission bandwidth or hard-disk space. Compression algorithms also take advantage of human perception to further reduce the size of information. By encoding the same piece of information with fewer bits in places that the human brain is not accustomed to perceiving as well as other parts, substantial gains in compression ratios can be achieved.

9175 questions
4
votes
3 answers

Manual alternative to mod_deflate

Say I don't have mod_deflate compiled into apache, and I don't feel like recompiling right now. What are the downsides to a manual approach, e.g. something like: AddEncoding x-gzip .gz RewriteCond %{HTTP_ACCEPT_ENCODING} gzip RewriteRule…
Bobby Jack
  • 15,689
  • 15
  • 65
  • 97
4
votes
6 answers

Compress array of numbers

I have a large array (~400.000.000 entries) with integers of {0, 1, ..., 8}. So I need 4 bits per entry. Around 200 MB. At the moment I use a byte-array and save 2 numbers in each entry. I wonder, if there is a good method, to compress this…
Jakube
  • 3,353
  • 3
  • 23
  • 40
4
votes
2 answers

In Linux, How to Compress Input Stream to Multiple Output Files?

I know how to do this: commandGeneratingLotsOfSTDOUT | bzip2 -z -c > compressed.bz2 I also know how to do this: commandGeneratingLotsOfSTDOUT | split -l 1000000 But I don't know how to do this: commandGeneratingLotsOfSTDOUT | split -l 1000000…
philo vivero
  • 427
  • 5
  • 16
4
votes
1 answer

Hive ORC compression

I run following code in hive v0.12.0 and I expect to get three tables compressed using different methods and therefore size and content of the files should be different. --- Create table and compress it with ZLIB create table zzz_test_szlib stored…
Andrey Dmitriev
  • 528
  • 2
  • 9
  • 27
4
votes
4 answers

encrypt a string to obtain a fixed length

let's suppose I have the following possible values of a string: exp="110" exp="110-120" exp="110-120-211" actually, this is a GET parameter obtained from the URL. exp can become very large so I want to make it shorter(encrypt it). Not because of…
Mihai Zamfir
  • 2,167
  • 4
  • 22
  • 37
4
votes
1 answer

Why are JPEG quantization matrices asymmetric?

Why are the basis blocks corresponding to reflected waves in the quantisation matrix given seemingly random priorities in the standard JPEG quantisation matrices. Also, why aren't the priorities monotonic with respect to frequency? I haven't been…
4
votes
1 answer

Decoding a compressed short string; uncertain on compression used - Updated

I have a program that is compressing a string in an unknown way. I know a few inputs and the output produced, but I am not sure what is being used to compress the string. Here are my examples. (just 38 x a, no spaces or anything else) In: …
James
  • 139
  • 1
  • 9
4
votes
1 answer

Compressing size of mp4 file

I'm trying to compress an mp4 file to reduce the file size. I am using a Debian box. The file is around 701 MB. First, I tried bzip2, which compressed it down to like 700MB, then I tried lrzip with zpaq, took like 5 mins to compress, and only…
user3180103
  • 43
  • 1
  • 4
4
votes
1 answer

Android: lossless WEBP compression

I am compressing an image in the WEBP-format with the following code: bitmap.compress(Bitmap.CompressFormat.WEBP, 100, outputStream) Although I am using highest quality-settings, it looks like the image is not compressed lossless but…
Alex
  • 569
  • 5
  • 21
4
votes
3 answers

How to unzip a file in IsolatedStorage in windows phone 8 app?

Inside my app, I am trying to download about 180 small audio files all at once. I tried the BackgroundTransferService, but it does not seem stable with so many small files. So, now I am downloading a ZIP of all those audio and want extract them in…
hnabbasi
  • 191
  • 9
4
votes
7 answers

Compression of numeric strings

Can anyone suggest compression algorithms to operate on numeric strings of 20-30 digits ?
Vijay Dev
  • 26,966
  • 21
  • 76
  • 96
4
votes
1 answer

Performance cost of serialization and compress a Object in Java

The application keeps receiving objects named Report and put the objects into Disruptor for three different consumers. With the help of Eclipse Memory Analysis, the Retained Heap Size of each Report object is 20KB on average. The application starts…
macemers
  • 2,194
  • 6
  • 38
  • 55
4
votes
2 answers

Efficient way to store list of URLs

I need to store trillion of list of URLs where each URL list will contain ~50 URLs. What would be the most space efficient way to compress them for on-disk storage. I was thinking of first removing useless information like "http://" and then build a…
skyde
  • 2,816
  • 4
  • 34
  • 53
4
votes
1 answer

Not in GZIP format Java

I'm trying to download .torrent files from the internet. Some of the files online are compressed (gzipped) format. I know I can unzip the files with the following code: try (InputStream is = new GZIPInputStream(website.openStream())) { …
Joe
  • 111
  • 3
  • 14
4
votes
1 answer

Lossless H.264 compression of a 16-bit grayscale image

I have a 16-bit monochrome image that I need to encode it using lossless H.264. Ideally, I would love to be able to use a 16-bit Y component, and then empty U and V components, but I believe such an implementation does not exist. On the other…
mhernandez
  • 607
  • 5
  • 14