2

I ran the following code to compute the pseudo inverse of a matrix, but it seems that it makes no difference whether I switch on the GPU or not.

mat = theano.shared(numpy.eye(300, dtype="float32")+1)
fn = theano.function([], theano.tensor.nlinalg.pinv(mat))
fn()

Then I looked at Theano's source code of theano.tensor.nlinalg.MatrixPinv, and found it just calls Numpy's numpy.linalg.pinv in the following code (I'll leave out the comments).

class MatrixPinv(Op):
    __props__ = ()

    def __init__(self):
        pass

    def make_node(self, x):
        x = as_tensor_variable(x)
        assert x.ndim == 2
        return Apply(self, [x], [x.type()])

    def perform(self, node, inputs, outputs):
        (x,) = inputs
        (z,) = outputs
        z[0] = numpy.linalg.pinv(x).astype(x.dtype)

pinv = MatrixPinv()

I'm not very familar with how Numpy is implemented, can it run on GPU?
If not, does that mean that every time I want to compute matrix inverse in Theano, I have to go back from GPU to CPU?

dontloo
  • 10,067
  • 4
  • 29
  • 50

1 Answers1

3

See the article Using the GPU in the theano docs.

Note that we use the shared function to make sure that the input x is stored on the graphics device.

You have to make sure that the data is stored in graphics memory. Otherwise, I imagine, theano falls back to using numpy routines.

Numpy normally does not run on the GPU. I'm not sure if it is possible to link it against CudaBLAS for doing so, but I guess that is out of scope here.

MB-F
  • 22,770
  • 4
  • 61
  • 116
  • Thanks for pointing out. I edited the code a bit, and it turned out to have similar results, so I guess it does not use GPU at all? – dontloo Jan 27 '16 at 02:04
  • Honestly, I don't know. Maybe the matrix inverse is not well suited for GPU use. But on the other hand, the inverse is equivalent to solving quations, which I belive can be done on the GPU. Maybe you have a configuration issue - do other operations use the gpu? – MB-F Jan 27 '16 at 07:31
  • A few more points to check: Do you have a nvidia card? Do you have CUDA installed? – MB-F Jan 27 '16 at 07:33
  • Thank you so much, and yes I have CUDA and other operations like matrix multiplication can run on the GPU well (though I'm not sure how to verify this, but it does get a lot faster), maybe Theano has not support GPU matrix inversion yet. – dontloo Jan 27 '16 at 08:30