Cython class initialization in ray

Question

I'm using Ray (https://github.com/ray-project/ray) and Cython 0.29 to parallelize some existing code, and I decided to define a cdef class with one of my Cython functions as its method to ease running the code for multiple Actors in paralell. The problem I've run into is that when I create an instance of my class following the cython_simple.py (example6), I get a "Too many arguments to __init__ error".

I have a few questions, but the first is how to correctly give my class instance its arguments when it gets decorated by ray.remote.

I've been trying this method where Test is the imported class:

    import ray
    from cython_test.cython_test import Test
    ray.init()
    Test1 = ray.remote(Test)
    #Instatiate an actor
    args = (img, 10, 10)
    a1 = Test1.remote(*args)

But I get back:

    Traceback (most recent call last):
      File "test_ray_cython.py", line 25, in <module>
        a1 = Test1.remote(*args)
      File "/local/data/home/gmosby/.local/lib/python3.7/site-packages/ray/actor.py", line 282, in remote
        return self._remote(args=args, kwargs=kwargs)
      File "/local/data/home/gmosby/.local/lib/python3.7/site-packages/ray/actor.py", line 384, in _remote
        kwargs)
      File "/local/data/home/gmosby/.local/lib/python3.7/site-packages/ray/signature.py", line 221, in extend_args
        .format(function_name))
    Exception: Too many arguments were passed to the function '__init__'

Additional Info: The Cython class here was initialized as def __init__(self, img, nx, ny)

Is it the case that if I want to use a Cython class with Ray it must not have any initialization arguments?

WORKAROUND found: It does appear decorating Cython classes with positional arguments in their __init__ methods fails, but I’ve decided to just move the setting of some class members to a separate function. See illustration of this issue and workaround here (https://github.com/astrophysaxist/cython_test)

Here are some docs: https://ray.readthedocs.io/en/releases-0.7.3/example-cython.html — richliaw, Sep 14 '19 at 04:00
The problem has nothing to do with Ray and nothing to do with Cython. A simpler [mre] would be: `class Test: def __init__(self): pass` then `Test(1,2,3)` - you have a constructor that wants one argument and you're passing it more — DavidW, Sep 14 '19 at 07:36
@DavidW I think this may probably arise from an unspoken Ray rule. It seems it's not acceptable to have Cython classes with positional arguments in their `__init__`. See https://github.com/astrophysaxist/cython_test. Once I removed the extra arguments to `__init__` from my Cython function remote was able to create the class instance no problem. — greg_pro, Sep 14 '19 at 20:51
In future it would be good to put this information in the question. The class you were using when you asked was a regular (rather than `cdef` class) and had a constructor that only took `self`. Your question __must__ demonstrate the problem you're reporting — DavidW, Sep 15 '19 at 08:00
Here's another illustration of this bug https://github.com/DerwenAI/rayCy_bug FWIW, I author and teach some of the tutorials for Ray, and this issue was not obvious. — Paco, Aug 21 '21 at 19:01

score 1 · Answer 1 · answered Sep 14 '19 at 22:53

WORKAROUND found: It does appear decorating Cython classes with positional arguments in their __init__ methods fails, but I’ve decided to just move the setting of some class members to a separate function. See illustration of this issue and workaround here (https://github.com/astrophysaxist/cython_test)

Cython class initialization in ray

1 Answers1