3

Is there a way to use ray with celery. Which executor to use. I've tried a toy implementation and ran into following error:

[2020-05-25 00:21:07,473: ERROR/ForkPoolWorker-5] Task mytasks.add[4a2f6fba-4f1b-4a77-95a0-0ee3c488a927] raised unexpected: AttributeError("'LoggingProxy' object has no attribute 'fileno'",)
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 382, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 641, in __protected_call__
return self.run(*args, **kwargs)
File "/home/akshay/Desktop/mytasks.py", line 15, in add
ray.init()
File "/home/akshay/.local/lib/python3.5/site-packages/ray/worker.py", line 1453, in init
driver_id=driver_id)
File "/home/akshay/.local/lib/python3.5/site-packages/ray/worker.py", line 1716, in connect
faulthandler.enable(all_threads=False)
AttributeError: 'LoggingProxy' object has no attribute 'fileno'

The Code implementation is here:

from celery import Celery, signals
import logging
import ray
celery = Celery('tasks', broker='redis://localhost:6379/0', backend='redis://localhost')
@celery.task
def add(value):
    ray.init()
    to = remote_chain_function.remote(value)
    return to


@ray.remote
def remote_chain_function(value):
    return value + 1
noobwithskills
  • 69
  • 1
  • 9
  • 1
    It is pretty interesting you tried this. Are you doing this in order to submit jobs to the Ray cluster from a remote node? – Sang May 25 '20 at 05:15
  • 1
    I think you should instantiate celery and ray in the same node first, and just use ray functions inside celery tasks without ray.init. And you can invoke the celery tasks from the remote nodes. If it doesn't work, can you post this on the Ray github issue? – Sang May 25 '20 at 05:18
  • Thanks @Sang. Yes the idea is to submit jobs to ray cluster. I'll work on your suggestion and get back to you on this. – noobwithskills May 26 '20 at 10:47
  • @Sang Seems the workaround you suggested doesn't work. I've posted it on GitHub issue. – noobwithskills May 26 '20 at 11:33

1 Answers1

0

You can get rid of the AttributeError: 'LoggingProxy' object has no attribute 'fileno' by setting this setting:

CELERY_WORKER_REDIRECT_STDOUTS = False

https://docs.celeryproject.org/en/stable/userguide/configuration.html?highlight=REDIRECT_STDOUTS#std-setting-worker_redirect_stdouts

keyvanm
  • 157
  • 9