How can i use a unsigned 64 bits variables in Bits Operations in Clojure?

Question

I have the following code:

(defn BitScanReverse [^Long bit-board]
  (loop [value bit-board r 0]
    (cond
      (> value 0x00000000FFFFFFFF) (recur (unsigned-bit-shift-right value 32) (+ r 32))
      (> value 0x000000000000FFFF) (recur (unsigned-bit-shift-right value 16) (+ r 16))
      (> value 0x00000000000000FF) (recur (unsigned-bit-shift-right value 8) (+ r 8))
      (> value 0x000000000000000F) (recur (unsigned-bit-shift-right value 4) (+ r 4))
      (> value 0x0000000000000003) (recur (unsigned-bit-shift-right value 2) (+ r 2))
      (> value 0x0000000000000001) (recur (unsigned-bit-shift-right value 1) (+ r 1))
      :else r)))

It returns the index of the last bit found in a bitboard. The problems is when I try to run: (BitScanReverse 18446462598732840960) ;;Expecting 63. Its gives me: IllegalArgumentException Value out of range for long: 18446462598732840960 clojure.lang.RT.longCast (RT.java:1134)

This bitboard is the initial position of the black pieces. The problem is that long is signed in clojure (and also in java). I've tried to use BigInt but it doesn't allow bit operations.

Any suggestions?

Answer 1

Here's a quick and very dirty implementation of a reverse scan using bit-test and no looping, which may or may not be more efficient.

(defn rev-test [^long n ^long x] (bit-test x n))
(defn BitScanReverse [^long bit-board](condp rev-test bit-board
0 0,1 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 11,12 12,13 13,14 14,15 15,16 16,17 17,18 18,19 19,20 20,21 21,22 22,23 23,24 24,25 25,26 26,27 27,28 28,29 29,30 30,31 31,32 32,33 33,34 34,35 35,36 36,37 37,38 38,39 39,40 40,41 41,42 42,43 43,44 44,45 45,46 46,47 47,48 48,49 49,50 50,51 51,52 52,53 53,54 54,55 55,56 56,57 57,58 58,59 59,60 60,61 61,62 62,63 63))

This treats the least-significant bit as 0, like bit-test does and like 0-indexed arrays do, so I don't think it's the same as your implementation. When it comes to producing input, you'll be limited to 63 bits with positive signed literals, but you can still use the sign bit as the 64th bit. Try making a helper method to construct the numbers you need with a slightly higher level of abstraction, like this fn that takes the most-significant 32 bits and the least-significant 32 bits as two args. This could probably be written as a macro, but I'm not experienced enough to write one and be sure it will work.

(defn bitboard [^long upper ^long lower]
    (bit-or (bit-shift-left upper 32)
            (bit-and lower 0xffffffff)))

Importantly for performance, ^Long is boxed, and I think ^long may not be boxed under the right circumstances. Numeric primitive arrays are one of the few cases I've found where primitives are assuredly what they are supposed to be on the JVM (a byte array will always be a byte array, with contiguous 8-bit pieces of memory, but a byte declared on its own, even in Java, can take up more than 8 bits of memory due to alignment optimization). I can strongly recommend ztellman's primitive-math library for finding cases where reflection is needed for math in Clojure, which is surprisingly often and can be very important for bit-twiddling code like this.

Answer 2

Using Java BitSet as @assylias suggests, gives a simple solution:

(import 'java.util.BitSet)

(defn bit-scan-reverse [^long bit-board]
  (let [board (BitSet/valueOf (long-array [bit-board]))]
    (.previousSetBit board 63)))

EDIT Above solution does not work since long is signed. Continuing the search with BitSet, I came up with:

(defn bit-scan-reverse2 [bit-board]
  (let [board (-> (biginteger bit-board) ; coerce to BigInteger
                  .toByteArray           ; as byte[] big-endian
                  reverse                ; to little-endian
                  byte-array             
                  BitSet/valueOf)
        max-index (.size board)]
    (.previousSetBit board max-index)))

Which works fine but seems quite complex. Then looking at BigInteger doc, I found the bitLength() method that actually answer the question:

bitLength(): Returns the number of bits in the minimal two's-complement representation of this BigInteger, excluding a sign bit.

Since we care only for positive numbers for the board representation, it's ok to use this bitLength() method to find the left-most bit set in the 64bit board:

(defn bit-scan-reverse3 [bit-board]
  (-> bit-board
      biginteger
      .bitLength
      dec))

(map bit-scan-reverse3 '(0xFFFF000000000000 0x0 0x1 0xF 0xFFF))
user> 63 -1 0 3 11

** END EDIT**

In term of performances, I tested the 3 versions, and it gives very similar timing (~10ns) for this quick-bench. BitScanReverse2 is the solution provided by @notostraca:

(require '[clojure.data.generators :as gen])
(require '[criterium.core :as c])

(let [bunch-of-longs (doall (take 10000 (filter (partial < 0) 
                                                (repeatedly gen/long))))]
  (c/quick-bench (map BitScanReverse bunch-of-longs))
  (c/quick-bench (map BitScanReverse2 bunch-of-longs))
  (c/quick-bench (map bit-scan-reverse bunch-of-longs))
  (c/quick-bench (map bit-scan-reverse2 bunch-of-longs))
  (c/quick-bench (map bit-scan-reverse3 bunch-of-longs)))