0

I am looking to create a custom scalar type that has magnitude and a bunch of sign bits. I have defined operations such as arithmetic and comparison. I want to use this scalar type with numpy extensively. (Eg., Normal operations would involve multiplying matrices of size 1000 X 1000 multiple times). So I need this to be as efficient as possible. I have an implementation in C++ using Eigen as matrix library but I need a python port to make it easily accessible in python.

I see three ways as of now :

The operations of my type are only bit level (addition, for) but they would be done millions of times.

I would like to know what method would create the most efficient scalar type.

  • writing a class in python Vs using boost, would there be any observable difference in speed
  • writing an extension type in C Vs class in python, would the difference be noticable.

I could carry out experiment but I am looking for some experience. If this doesn't get answered, I will answer myself.

Edit:

As suggested by @Parfait, it looks like the best way would be using Cython cdef. Here is a quick benchmark involving numpy and Cython.

Aditya Singh Rathore
  • 1,016
  • 6
  • 23
  • 1
    Another option is extending numpy [doc](https://numpy.org/doc/stable/user/c-info.how-to-extend.html). I've never done this myself, but you want to be looking at vectorized operations on primitive types. If you create a python object, that object will be in the numpy array. It will be large (python objects have a minimum size and ref count) and operations will go through its relatively slow python method calls. – tdelaney May 25 '21 at 16:07
  • Consider another route with Cython where you can define your type, integrate it in a cdef class, and compile code to be available as a Python module for `import` call. – Parfait May 25 '21 at 16:33
  • @tdelaney I had thought of that. But as I mentioned, I have overloaded * and + to behave differently. I would have to write matrix multiplication myself. – Aditya Singh Rathore May 25 '21 at 16:39
  • 1
    I've used boost, swig, cython shims and etc... but really, just writing the extension in C [Extending and Embedding the Python Interpreter](https://docs.python.org/3/extending/index.html) is the fastest and easiest way to glue in existing C++ code for someone who is already a C/C++ programmer. Everything else complicates the matter IMHO. – tdelaney May 25 '21 at 16:59

0 Answers0