Questions tagged [compression]

The name given to the process of encoding data such that it uses lesser number of bits as compared to the original representation.

There are a number of techniques to compress information, with most algorithms working to compress image, speech, music or video. Encoding the same information in lesser bits results in saving some crucial resource such as transmission bandwidth or hard-disk space. Compression algorithms also take advantage of human perception to further reduce the size of information. By encoding the same piece of information with fewer bits in places that the human brain is not accustomed to perceiving as well as other parts, substantial gains in compression ratios can be achieved.

9175 questions
23
votes
2 answers

Is it possible to compress video on Android?

I want to do video compression. Actually in my application I want to have two options, one is low and another is high. If I choose the low option then the application will compress the video and then upload it. If I choose high then it will upload…
Jay Thakkar
  • 743
  • 1
  • 5
  • 24
22
votes
3 answers

How to see contents of Hive orc files in linux

Is there a way to see the contents of an orc file that hive 0.11 and above use. I usually cat gz files and decompress them to see the contents eg: cat part-0000.gz | pigz -d | more Note: pigz is a parallel gz program. I would like to know if there…
viper
  • 2,220
  • 5
  • 27
  • 33
22
votes
4 answers

Spark Standalone Mode: How to compress spark output written to HDFS

Related to my other question, but distinct: someMap.saveAsTextFile("hdfs://HOST:PORT/out") If I save an RDD to HDFS, how can I tell spark to compress the output with gzip? In Hadoop, it is possible to set mapred.output.compress = true and choose…
ptikobj
  • 2,690
  • 7
  • 39
  • 64
21
votes
9 answers

ZLIB Decompression - Client Side

I am receiving data as an "ZLIB" compressed inputstream. Using Javascript/Ajax/JQuery, I need to uncompress it on the client side. Is there a way to do so? I already have this working in JAVA as below, but need to do this on Client Side. url = new…
Rajeev Raina
  • 239
  • 1
  • 3
  • 8
21
votes
4 answers

Any theoretical limit to compression?

Imagine that you had all the supercomputers in the world at your disposal for the next 10 years. Your task was to compress 10 full-length movies losslessly as much as possible. Another criteria was that a normal computer should be able to decompress…
David
  • 4,786
  • 11
  • 52
  • 80
21
votes
2 answers

Why do the md5 hashes of two tarballs of the same file differ?

I can run: echo "asdf" > testfile tar czf a.tar.gz testfile tar czf b.tar.gz testfile md5sum *.tar.gz and it turns out that a.tar.gz and b.tar.gz have different md5 hashes. It's true that they're different, which diff -u a.tar.gz b.tar.gz…
user3721976
21
votes
2 answers

How do you unzip very large files in python?

Using python 2.4 and the built-in ZipFile library, I cannot read very large zip files (greater than 1 or 2 GB) because it wants to store the entire contents of the uncompressed file in memory. Is there another way to do this (either with a…
Marc Novakowski
  • 44,628
  • 11
  • 58
  • 63
21
votes
2 answers

analysing packed file of unknown format, how to continue?

Im interested in taking a deeper look into the firmware of my Behringer X32 mixing console, so i downloaded the actual file from (http://www.behringerdownload.de/X32/X32_Firmware_2.10.zip) and started IDA pro. The contained .update-file inside…
rhavin
  • 1,512
  • 1
  • 12
  • 33
21
votes
6 answers

Compressing a folder with many duplicated files

I have a pretty big folder (~10GB) that contains many duplicated files throughout it's directory tree. Many of these files are duplicated up 10 times. The duplicated files don't reside side by side, but within different sub-directories. How can I…
user972014
  • 3,296
  • 6
  • 49
  • 89
21
votes
1 answer

How to compress files into zip folder in android?

I am trying to send multiple files in email, i got the codes for attaching them in the email. Intent sendIntent = new Intent(Intent.ACTION_SEND_MULTIPLE); sendIntent.setType("application/octet-stream"); ArrayList
Vicky
  • 921
  • 1
  • 11
  • 33
20
votes
10 answers

What is the best compression algorithm for small 4 KB files?

I am trying to compress TCP packets each one of about 4 KB in size. The packets can contain any byte (from 0 to 255). All of the benchmarks on compression algorithms that I found were based on larger files. I did not find anything that compares the…
David
20
votes
5 answers

javascript string compression with localStorage

I am using localStorage in a project, and it will need to store lots of data, mostly of type int, bool and string. I know that javascript strings are unicode, but when stored in localStorage, do they stay unicode? If so, is there a way I could…
invisible bob
  • 864
  • 1
  • 9
  • 24
20
votes
1 answer

What is urlCompression in IIS7?

In my ASP.NET web.config, I have this: I went here to find an answer: http://www.iis.net/ConfigReference/system.webServer/urlCompression ...but it doesn't really explain what URL compression is. Can…
oscilatingcretin
  • 10,457
  • 39
  • 119
  • 206
20
votes
4 answers

How does MPEG4 compression work?

Can anyone explain in a simple clear way how MPEG4 works to compress data. I'm mostly interested in video. I know there are different standards or parts to it. I'm just looking for the predominant overall compression method, if there is one with…
jeffD
  • 1,230
  • 3
  • 13
  • 18
20
votes
6 answers

How do I use UPX with pyinstaller?

How do I use UPX with pyinstaller? I am following the docs. I have downloaded UPX. My file looks like: import csv import selenium import pandas print('Hello') I then run: pyinstaller -F --upx-dir…
user9062171