1

I have an example of a string ABABABAB and I have to calculate an entropy of this string.

It's obvious that i can get different numbers when taking different alphabets. When I took alphabet A={a,b} I got an answer for entropy = 1 bit per character(Using Shannon's formula) => means 8 bits for a whole string.

But what about a case when we take A={ab,aa,bb,ba}?

We get entropy =0 bits per character (which is also obvious,as there is no randomness). How is it possible to have 0 bpc ? So the whole string = 0 bits?=/ I can't understand where I got wrong..

Thanks in advance for any kind of help.

Community
  • 1
  • 1
I K
  • 11
  • 2

1 Answers1

0

Yes, it's possible, but other information needs to be sent. In particular that there are four encoded symbols, and that the only possible symbol is AB. Once you've sent those things, the remainder is zero bits.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158