Python: Extracting bits from a byte

Question

I'm reading a binary file in python and the documentation for the file format says:

Flag (in binary)Meaning

1 nnn nnnn Indicates that there is one data byte to follow that is to be duplicated nnn nnnn (127 maximum) times.

0 nnn nnnn Indicates that there are nnn nnnn bytes of image data to follow (127 bytes maximum) and that there are no duplications.

n 000 0000 End of line field. Indicates the end of a line record. The value of n may be either zero or one. Note that the end of line field is required and that it is reflected in the length of line record field mentioned above.

When reading the file I'm expecting the byte I'm at to return 1 nnn nnnn where the nnn nnnn part should be 50.

I've been able to do this using the following:

flag = byte >> 7
numbytes = int(bin(byte)[3:], 2)

But the numbytes calculation feels like a cheap workaround.

Can I do more bit math to accomplish the calculation of numbytes?

How would you approach this?

related: [Using Python How can I read the bits in a byte?](http://stackoverflow.com/q/2576712/4279) — jfs, May 12 '16 at 14:18

Zaur Nasibov · Answer 1 · 2018-07-25T15:29:56.483

27

The classic approach of checking whether a bit is set, is to use binary "and" operator, i.e.

x = 10 # 1010 in binary
if x & 0b10:  # explicitly: x & 0b0010 != 0
    print('First bit is set')

To check, whether n^th bit is set, use the power of two, or better bit shifting

def is_set(x, n):
    return x & 2 ** n != 0 

    # a more bitwise- and performance-friendly version:
    return x & 1 << n != 0

is_set(10, 1) # 1 i.e. first bit - as the count starts at 0-th bit
>>> True

edited Jul 25 '18 at 15:29

answered Mar 30 '12 at 15:16

Zaur Nasibov

22,280
12
56
83

Thanks, but that doesn't really answer my question. Given byte = 178, how would you extract flag == 1 & numbytes == 50? – Evan Borgstrom Mar 30 '12 at 15:18
1

+1: but one doesn't need the !=0 part, `if x&0b10` should be sufficient. – tom10 Mar 30 '12 at 15:21
2

`!= 0` is not quite as bad as `== True`, but it is close. :) – Sven Marnach Mar 30 '12 at 15:22
3

[_Explicit is better than implicit_](http://www.python.org/dev/peps/pep-0020/) so `x & 0b10 != 0` is fine by me – D.Shawley Mar 30 '12 at 15:23
Consider this: bin(178) == 10110010; If I uderstood correctly, you need to return all bytes starting from i.e. 5th byte. Then 178 & (2^5 - 1), should do the trick, because in this case 10110010 & 00011111 == 00010010 – Zaur Nasibov Mar 30 '12 at 15:25
1

`print('First bit is set')` – the word *first* is **very** confusing here. – Wolf Aug 05 '22 at 09:15

alan · Accepted Answer · 2020-02-24T21:54:08.570

18

You can strip off the leading bit using a mask ANDed with a byte from file. That will leave you with the value of the remaining bits:

mask =  0b01111111
byte_from_file = 0b10101010
value = mask & byte_from_file
print bin(value)
>> 0b101010
print value
>> 42

I find the binary numbers easier to understand than hex when doing bit-masking.

EDIT: Slightly more complete example for your use case:

LEADING_BIT_MASK =  0b10000000
VALUE_MASK = 0b01111111

values = [0b10101010, 0b01010101, 0b0000000, 0b10000000]

for v in values:
    value = v & VALUE_MASK
    has_leading_bit = v & LEADING_BIT_MASK
    if value == 0:
        print "EOL"
    elif has_leading_bit:
        print "leading one", value
    elif not has_leading_bit:
        print "leading zero", value

edited Feb 24 '20 at 21:54

answered Mar 30 '12 at 15:23

alan

4,752
21
30

Thanks. I too prefer the binary numbers in this case. – Evan Borgstrom Mar 30 '12 at 15:31
3

@EvanBorgstrom On Python 3, you could write: `data = b'\xaa\x55\x00\x80'` and `for byte in data: ...`. Don't use `bytes` name, it is a builtin type. – jfs May 12 '16 at 14:25

score 1 · Answer 3 · answered Mar 30 '12 at 15:21

1

If I read your description correctly:

if (byte & 0x80) != 0:
    num_bytes = byte & 0x7F

answered Mar 30 '12 at 15:21

D.Shawley

58,213
10
98
113

score 1 · Answer 4 · answered Mar 30 '12 at 15:28

there you go:

class ControlWord(object):
    """Helper class to deal with control words.

    Bit setting and checking methods are implemented.
    """
    def __init__(self, value = 0):
        self.value = int(value)
    def set_bit(self, bit):
        self.value |= bit
    def check_bit(self, bit):
        return self.value & bit != 0
    def clear_bit(self, bit):    
        self.value &= ~bit

score 1 · Answer 5 · answered Mar 30 '12 at 15:28

1

Instead of int(bin(byte)[3:], 2), you could simply use: int(bin(byte>>1),2)

answered Mar 30 '12 at 15:28

theonova

21
2

score 0 · Answer 6 · answered Mar 30 '12 at 15:19

0

not sure I got you correctly, but if I did, this should do the trick:

>>> x = 154 #just an example
>>> flag = x >> 1
>>> flag
1
>>> nb = x & 127
>>> nb
26

answered Mar 30 '12 at 15:19

Not_a_Golfer

47,012
14
126
92

score 0 · Answer 7 · answered Mar 30 '12 at 15:27

You can do it like this:

def GetVal(b):
   # mask off the most significant bit, see if it's set
   flag = b & 0x80 == 0x80
   # then look at the lower 7 bits in the byte.
   count = b & 0x7f
   # return a tuple indicating the state of the high bit, and the 
   # remaining integer value without the high bit.
   return (flag, count)

>>> testVal = 50 + 0x80
>>> GetVal(testVal)
(True, 50)

Python: Extracting bits from a byte

7 Answers7