I need to use(in an application) Extreme Learning Machine(ELM) which is highly optimised for multiple CPUs or GPUs. As ELM main computation involves Moore-Penrose Pseudo inverse and matrix multiplication, what would be the best option to implement ELM among Theano and Caffe ?
Secondly, Is it possible to implement a new learning algorithm(ELM) in Caffe using its python interface ?
If I were you I would use Theano, not Caffe. Caffe is not programmed around a general-purpose matrix library so with Caffe you would be trying to use a screwdriver to open a beer basically. If you definitively feel like using C++ look into MrShadow or any other GPU-based matrix libraries.
... or simply use Theano with Python.
I'm not a big fan of Python and Theano takes some time to master but it's extremely convenient.
Also there are one or two ELM libraries for Python you can use as a reference, that's a huge plus when you need to test your own implementation.
https://github.com/dclambert/Python-ELM
I haven't used them so I can't elaborate on their status but something is better than nothing.
You can also take a look at Keras and Lasagne, both are neural network libraries built on top of Theano. Just like Caffe, they will not help much with ELMs but they will get you started with Theano+nnets. Then all you have to do is create your own ELM layers.
An alternative solution is PyTorch's torch.linalg.pinv(), optimised for GPU computations:
Computes the pseudo-inverse (also known as the Moore-Penrose inverse) of a matrix
input, or of each matrix in a batchedinput.
As far as Google is concerned Caffe won't help you with "Extreme Learning Machines".
Secondly, Is it possible to implement a new learning algorithm(ELM) in Caffe using its python interface ?
No, that is not possible. You will have to implement new layers and algorithms in C++. Afterwards you can deal with them via Python.
For a primer on Caffe, check out "Neural Nets with Caffe Utilizing the GPU".
