Exploiting Symmetry in Large Numpy Array

Question

Want to make large array, B, that is made up of a smaller numpy array, A, flipped in different ways:

B[0,:,:,:,:]   = A
B[1,:,:,:,:]   = B[0,:,::-1,:,:]
B[2:4,:,:,:,:] = B[0:2,:,:,::-1,:]
B[4:8,:,:,:,:] = B[0:4,:,:,:,::-1]

Is there a way to only store A in memory, but keep some of the functionality of a numpy array for B? I was primarily interested in two things:

Be able to scale B[m,n,...] (i.e. B[m,n,...] *= C where B.shape[2:] == C.shape)
Be able sum down to the second dimension (i.e. np.sum(B, axis=(2,3,4)))

What functionality do you need? A large range of operations are likely to destroy your symmetry (and therefore any gains you might get by exploiting the symmetry).... — mgilson, May 03 '16 at 15:01
In the two-dimensional case, there is the concept of block-matrices which would allow you to perform particular operations on parts of the matrix and cache the results of those operations. Thus, when you combine the operations on the individual blocks, you can perform them once and use the cached version. I'm sure there is some generalization to tensors, but you'd have to do some research. So, this is certainly possible, but as @mgilson points out, it will only be useful for particular operations. — Alex Alifimoff, May 03 '16 at 16:04
@mgilson, you're right, so I updated the question with the main functionality I was looking for. — dfreese, May 03 '16 at 17:18
When you do your first operation, should the reflections of A be scaled by identically-reflected copies of C or the unreflected C? — user2357112, May 03 '16 at 18:00
@dfreese: Just to be clear, scaling the reflections by the unreflected C would break the symmetry. Should the scaling operation be equivalent to scaling A and then creating B from the scaled A, or should it be inequivalent? — user2357112, May 03 '16 at 20:25
I realize that I should have described it as being able to scale B[m,n,...] by C. In that case I will be breaking symmetry, so the most useful thing would be to have a structure to access B[m,n, ...] and be able to scale afterwards. Views of A should be sufficient for that purpose. — dfreese, May 04 '16 at 18:34

score 0 · Accepted Answer · answered May 05 '16 at 16:52

What I ended up doing was creating a class to return a view of an arbitrarily reflected portion of A. I was doing scaling by C, as well as sums, after returning this view, which seemed quick enough for now. This is it without error checking:

class SymmetricArray:
    '''
    Attributes:
        A (np.ndarray): an [m, (a,b,c...)] array.
        idx (np.ndarray): an [n,] array where idx[n] points to A[idx[n], ...]
            to be used.
        reflect (np.ndarray): an [n, d] array where every entry is 1 or -1 and
            reflect[n, i] indicates whether or not the ith dimension of
            A[idx[n], ...] should be reflected, and d = len(A.shape - 1).
    '''
    def __init__(self, A, idx, reflect):
        self.A = np.asarray(A)
        self.idx = np.asarray(idx, dtype=int)
        self.reflect = np.asarray(reflect, dtype=int)

    def __getitem__(self, ii):
        '''
        Returns:
            np.ndarray: appropriately reflected A[idx[ii], ...]
        '''
        a_mask = [slice(None, None, a) for a in self.reflect[ii, :]]
        return self.A[self.idx[ii], ...][a_mask]

Exploiting Symmetry in Large Numpy Array

1 Answers1