0

If I have a set of symbols and frequencies: A - 0.1 B - 0.40 C - 0.2 D - 0.23 E - 0.15 F - 0.17

The Huffman algorithm will produce codewords that are only greater than length 1.

But when I change a frequency to be greater than 0.40, it will produce a codeword of length 1 and greater. How can construct a proof that proves that this is the case for any set of symbols, not just this one?

Catherine Pierce
  • 53
  • 2
  • 7

2 Answers2

3

(Note that your frequencies don't add to 1; I'll assume it's a typo)

Here is a sketch of a proof that to make all codewords greater than 1 bit, no frequency can be greater than 2/5. Without loss of generality, the huffman tree must look like this:

    a+b+c+d (the sum must be equal to 1)
     /   \
  a+b     c+d
  / \     / \
 a   b   c   d

We must prove that all of a, b, c, and d are no greater than 2/5.

WLOG (again) a = b <= c <= d.

    2a+c+d
     /   \
   2a     c+d
  / \     / \
 a   a   c   d

Let's find the maximal value of d that is consistent with this Huffman tree. According to how the algorithm works, the following inequalities hold:

  • a <= c
  • a <= d
  • 2a >= c
  • 2a >= d

Let's also replace c by 1-d-2a:

  • a <= (1-d)/3
  • a <= d
  • a >= (1-d)/4
  • a >= d/2

It's not immediately obvious how this constrains a and d, but you can easily plot the constraints in the a/d coordinate space. Then, you know which two of the above four inequalities are most important:

d/2 <= a <= (1-d)/3

From here:

d/2 <= (1-d)/3

So d <= 2/5.

anatolyg
  • 26,506
  • 9
  • 60
  • 134
  • Convincing, but.. surprising. I spent 10 minutes wondering why I couldn't construct a counter-example :) – harold Sep 21 '14 at 14:33
  • Can you also give me an idea of how to prove that if some symbol in the set has frequency greater than 0.4, that huffman's algo will produce a codeword of length 1. – Catherine Pierce Sep 21 '14 at 16:47
  • I don't think you proved whatever it was you thought you proved. See my examples. – Mark Adler Sep 21 '14 at 20:43
  • I understood the question this way: "if all codes are longer than 1, then *whatever*". You (probably) understood it in another way: "if one code has length 1, then *whatever*". It doesn't matter whether my understanding is correct, because it leads to a fun exercise. – anatolyg Sep 22 '14 at 07:33
0

If you have three symbols with any frequencies, you will get one code of length 1 and two codes of length 2. They could, for example, all have probability 1/3, which is less than 0.4.

Here is a simple counter-example to the assertion with four symbols and their probabilities resulting in a code of length 1, where all probabilities are less than 0.4:

a - 0.34
b - 0.33
c - 0.17
d - 0.16

It is easy to construct longer codes with the same property, by simply breaking up the probabilities. E.g.:

a - 0.34
b - 0.33
c - 0.17
d - 0.08
e - 0.08
Mark Adler
  • 101,978
  • 13
  • 118
  • 158