Huffman coding is based on what Greedy Approach or Dynamic Programming

Question

Can we solve Problem of Huffman Coding by using Dynamic Programming, Is there any algorithm

score 3 · Answer 1 · answered May 12 '13 at 19:28

Huffman Coding works by creating a binary tree of nodes. These can be stored in a regular array, the size of which depends on the number of symbols, n. Huffman Coding can be implemented in O(n logn) time by using the Greedy Algorithm approach. Huffman Coding is not suitable for a Dynamic Programming solution as the problem does not contain overlapping sub problems.

score 2 · Accepted Answer · 2013-06-07T08:24:29.103

2

As per my knowledge about algorithms, I beleive huffman coding is based on Greedy approach. As we do in Activity selection problem. Task scheduling and knapsack problem are another examples of it.

edited Jun 07 '13 at 08:24

answered May 12 '13 at 19:17

1

IIRC the standard approach is to use a priority queue (e.g. binary heap). Low probability means high priority. Fill the queue with the initial leaf nodes for your alphabet. The take out the two highest priority (ie lowest probability). Add a node with those as children, with it's probability as the sum, then add that new subtree back to the priority queue. Repeat until there's only one item left in the queue. – May 12 '13 at 22:08
1

I'm guessing that choosing at each step the two least existing subtrees to form the least possible new subtree is considered greedy because at each step it maximises the [entropy](https://en.wikipedia.org/wiki/Entropy_%28information_theory%29) of the next set of subtrees. – May 12 '13 at 22:10

vikky.rk · Answer 3 · 2016-02-01T09:18:54.233

[Update]: I think the solution I mentioned below generates an optimal prefix code, but not necessarily same as Huffman code. Huffman code, by definition, is generated by taking the greedy approach. While it is optimal, it is not the only solution. (For example, once the Huffman tree is generated, you could interchange the leaves in the same level to give them different codes while still being optimal)

I think this can be solved using Dynamic Programming, although it won't be that efficient. The approach here is very similar to finding the Optimal Binary Search Tree, since as you go one level down you add one more bit to the leaves.

Here is the link for the code to calculate the minimum number of total bits. Of course this is exponential, but if you use DP it can be solved in O(n^2) time. I haven't tried generating the encoding, but should be possible I believe.

Huffman coding is based on what Greedy Approach or Dynamic Programming

3 Answers3