Efficient way to bit mask every number of a file list using bitwise operators

Question

I have a file which contains a list of numbers defined as follow :

var1=0x00000001
var2=0x00000002
var3=0x00000008
var4=0x00000020
var5=0x00000040
var6=0x00000080
var7=0x00000100
var8=0x00000200
var9=0x00000400
var10=0x00000800 
var11=0x000001000
var12=0x000002000
var13=0x000004000
var14=0x000008000 
var15=0x00010000   
var16=0x00020000  
var17=0x00040000   
var18=0x10000000    
var19=0x20000000  
var20=0x40000000   
var21=0x80000000

I want to write something like this:

decValue=2147483650
printf -v hexValue "%x" "${decValue}"
echo $hexValue
IFS="="     
while read name ID x        
do
    test $((${hexValue} & ${ID})) = 0 && continue
    array+=("${name}")
done < "$FILE_NAME"

It returns :

80000002
var2 var9 var11 var12 var14 var17

But, in this specific case I just would like to return :

var21 var2

Other example, if decValue=12288 I would like to return var11 and var12.

Bitwise operators is a good tool to solve this issue ?

Wintermute · Accepted Answer · 2015-03-17T17:52:07.397

2

Use

printf -v hexValue "%#x" "${decValue}"

(or use ${decimalValue} in the test inside the loop)

As it is now, $hexValue ends up being 80000002 (as your own echo statement shows), and this is later interpreted as a decimal number when you want it to be interpreted as a hexadeximal one.

Passing %#x as format specifier to printf will make $hexValue have the value 0x80000002.

You'll also have to take another good look at the var table; there are a number of gaps in it. 0x4 is missing between var2 and var3, 0x10 is missing between var3 and var4, and between var17 and var18, the whole block from 0x80000 to 0x8000000 is gone as well. You're not going to get the results you expect for variables that have any of these bits set.

It might also be worth a thought to generate the bitmasks on the fly rather than holding them precalcuated in a file. One possible approach for that is

for((i = 0; (1 << i) <= $hexValue; ++i))
do
    test $(($hexValue & (1 << i))) = 0 && continue

    # Note: this will remember (zero-based) bit numbers rather than variable
    # names because there are no named variables any longer
    array+=($i)
done

In this, the bitshift expression 1 << i gives the number 2ⁱ, or put another way: 1 << i has the same value as var$((i + 1)) would in a repaired table.

edited Mar 17 '15 at 17:52

answered Mar 17 '15 at 16:56

Wintermute

42,983
5
77
80

This is a very good remark, I though resolve the problem. But it still does not work. If decValue=50332160 the script return only var8 and not var8 var18 var19 as I want to retrieve – ogs Mar 17 '15 at 17:28
That's because you put an extra zero on the right-hand side of vars 18 through 21. Actually, that list looks somewhat broken in the middle altogether. There are two extra zeros on the right hand side of those variables, but if you remove both, you'll not have enough variables to get the high bits of 50332160 -- you'll have to make the list longer to fix that. Or calculate the bitmasks on the fly. – Wintermute Mar 17 '15 at 17:34
There's a big missing chunk in your bitmask table between var17 and var18, that's what it boils down to. And also one between var2 and var3, and between var3 and var4. – Wintermute Mar 17 '15 at 17:39
thank for all your suggestions but the table is not defined by my own service. This is the reason why I cannot exchange their values. – ogs Mar 17 '15 at 22:03
If the bitmasks for the bits in your value are not in the table, are you even supposed to test for those bits? – Wintermute Mar 17 '15 at 22:17
decValue is set by an other function and is equal to 0 or an other random value, I have to test for those bits whatever happens.. that's probably the problem – ogs Mar 17 '15 at 22:32
Well, the bits in `var18` and `var19` are not set in `50332160`, so it wouldn't make sense for them to be part of the result array. The bits that are set in `50332160` that you don't catch are not part of any bitmask in your table because of those gaps, so I don't see by what logic you'd want what as part of the result array. – Wintermute Mar 17 '15 at 22:42

score 1 · Answer 2 · answered Mar 17 '15 at 18:04

1

Or use perl as:

perl -F= -slnE 'say $_ if( hex($F[1]) & $num )' -- -num=12288 < file

prints:

var11=0x000001000
var12=0x000002000

or

perl -F= -slnE 'say $F[0] if( hex($F[1]) & $num )' -- -num=12288 < file

prints

var11
var12

answered Mar 17 '15 at 18:04

clt60

62,119
17
107
194

Efficient way to bit mask every number of a file list using bitwise operators

2 Answers2