Fast computation of matrix rank over GF(2)

Question

For my current project I need to be able to calculate the rank of 64*64 matrices with entries from GF(2). I was wondering if anyone had a good solution.

I've been using pyfinite for this, but it is rather slow since it's a pure python implementation. I've also tried to cythonise the code I've been using, but have had issues due to relying on pyfinite.

My next idea would be to write my own class in cython, but that seems a bit overkill for what I need.

I need the following functionality

matrix = GF2Matrix(size=64) # creating a 64*64 matrix
matrix.setRow(i, [1,0,1....,1]) # set row using list
matrix += matrix2 # addition of matrices
rank(matrix) # then computing the rank

Thanks for any ideas.

Answer 1

One way to efficiently represent a matrix over GF(2) is to store the rows as integers, interpreting each integer as a bit-string. So for example, the 4-by-4 matrix

[0 1 1 0]
[1 0 1 1]
[0 0 1 0]
[1 0 0 1]

(which has rank 3) could be represented as a list [6, 13, 4, 9] of integers. Here I'm thinking of the first column as corresponding to the least significant bit of the integer, and the last to the most significant bit, but the reverse convention would also work.

With this representation, row operations can be performed efficiently using Python's bitwise integer operations: ^ for addition, & for multiplication. Then you can compute the rank using a standard Gaussian elimination approach.

Here's some reasonably efficient code. Given a collection rows of nonnegative integers representing a matrix as above, we repeatedly remove the last row in the list, and then use that row to eliminate all 1 entries from the column corresponding to its least significant bit. If the row is zero then it has no least significant bit and doesn't contribute to the rank, so we simply discard it and move on.

def gf2_rank(rows):
    """
    Find rank of a matrix over GF2.

    The rows of the matrix are given as nonnegative integers, thought
    of as bit-strings.

    This function modifies the input list. Use gf2_rank(rows.copy())
    instead of gf2_rank(rows) to avoid modifying rows.
    """
    rank = 0
    while rows:
        pivot_row = rows.pop()
        if pivot_row:
            rank += 1
            lsb = pivot_row & -pivot_row
            for index, row in enumerate(rows):
                if row & lsb:
                    rows[index] = row ^ pivot_row
    return rank

Let's run some timings for random 64-by-64 matrices over GF2. random_matrices is a function to create a collection of random 64-by-64 matrices:

import random

def random_matrix():
    return [random.getrandbits(64) for row in range(64)]

def random_matrices(count):
    return [random_matrix() for _ in range(count)]

and here's the timing code:

import timeit

count = 1000
number = 10
timer = timeit.Timer(
    setup="ms = random_matrices({})".format(count),
    stmt="[gf2_rank(m.copy()) for m in ms]",
    globals=globals())
print(min(timer.repeat(number=number)) / count / number)

The result printed on my machine (2.7 GHz Intel Core i7, macOS 10.14.5, Python 3.7) is 0.0001984686384, so that's a touch under 200µs for a single rank computation.

200µs is quite respectable for a pure Python rank computation, but in case this isn't fast enough, we can follow your suggestion to use Cython. Here's a Cython function that takes a 1d NumPy array of dtype np.uint64, again thinking of each element of the array as a row of your 64-by-64 matrix over GF2, and returns the rank of that matrix.

# cython: language_level=3, boundscheck=False

from libc.stdint cimport uint64_t, int64_t

def gf2_rank(uint64_t[:] rows):
    """
    Find rank of a matrix over GF2.

    The matrix can have no more than 64 columns, and is represented
    as a 1d NumPy array of dtype `np.uint64`. As before, each integer
    in the array is thought of as a bit-string to give a row of the
    matrix over GF2.

    This function modifies the input array.
    """
    cdef size_t i, j, nrows, rank
    cdef uint64_t pivot_row, row, lsb

    nrows = rows.shape[0]

    rank = 0
    for i in range(nrows):
        pivot_row = rows[i]
        if pivot_row:
            rank += 1
            lsb = pivot_row & -pivot_row
            for j in range(i + 1, nrows):
                row = rows[j]
                if row & lsb:
                    rows[j] = row ^ pivot_row

    return rank

Running equivalent timings for 64-by-64 matrices, now represented as NumPy arrays of dtype np.uint64 and shape (64,), I get an average rank-computation time of 7.56µs, over 25 times faster than the pure Python version.

Answer 2

I wrote a Python package galois that extends NumPy arrays over Galois fields. Linear algebra on Galois field matrices is one of the intended use cases. It is written in Python but JIT compiled using Numba for speed. It is quite fast and most linear algebra routines are also compiled. (One exception, as of 08/11/2021 the row reduction routine hasn't been JIT compiled, but that could be added.)

Here is an example using the galois library to do what you are describing.

Create a GF(2) array class and create an explicit array and a random array.

In [1]: import numpy as np                                                                                                                                                                     

In [2]: import galois                                                                                                                                                                          

In [3]: GF = galois.GF(2)                                                                                                                                                                      

In [4]: A = GF([[0, 0, 1, 0], [0, 1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0]]); A                                                                                                                                                                
Out[4]: 
GF([[0, 0, 1, 0],
    [0, 1, 1, 1],
    [1, 0, 1, 0],
    [1, 0, 1, 0]], order=2)

In [5]: B = GF.Random((4,4)); B                                                                                                                                                                
Out[5]: 
GF([[1, 1, 1, 0],
    [1, 1, 1, 0],
    [1, 1, 0, 0],
    [0, 0, 1, 0]], order=2)

You can update an entire row (as you requested) like this.

In [6]: B[0,:] = [1,0,0,0]; B                                                                                                                                                                  
Out[6]: 
GF([[1, 0, 0, 0],
    [1, 1, 1, 0],
    [1, 1, 0, 0],
    [0, 0, 1, 0]], order=2)

Matrix arithmetic works with normal binary operators. Here is matrix addition and matrix multiplication.

In [7]: A + B                                                                                                                                                                                  
Out[7]: 
GF([[1, 0, 1, 0],
    [1, 0, 0, 1],
    [0, 1, 1, 0],
    [1, 0, 0, 0]], order=2)

In [8]: A @ B                                                                                                                                                                                  
Out[8]: 
GF([[1, 1, 0, 0],
    [0, 0, 0, 0],
    [0, 1, 0, 0],
    [0, 1, 0, 0]], order=2)

There is an added method to the NumPy arrays called row_reduce() which performs Gaussian elimination on the matrix. You can also call the standard NumPy linear algebra functions on a Galois field array and get the correct result.

In [9]: A.row_reduce()                                                                                                                                                                         
Out[9]: 
GF([[1, 0, 0, 0],
    [0, 1, 0, 1],
    [0, 0, 1, 0],
    [0, 0, 0, 0]], order=2)

In [10]: np.linalg.matrix_rank(A)                                                                                                                                                              
Out[10]: 3

Hope this helps! If there is additional functionality desired, please open an issue on GitHub.