Running ray serve in a docker container in foreground, not daemon mode

Question

I'm running Ray Serve to host a HTTP API of a ray remote function. Is there a better way than below to run Ray Serve in foreground (i.e not daemon mode). Code is taken pretty straight from ray serve example:

import os
import time

import ray.serve

ray.serve.init(blocking=True, http_host="0.0.0.0", ray_init_kwargs={
    'webui_host': '0.0.0.0',
    'redis_password': os.getenv('RAY_REDIS_PASSWORD'),
})

ray.serve.create_endpoint("my_endpoint", "/echo")


def echo_v1(flask_request, response="hello from python!"):
    return "1"


ray.serve.create_backend(echo_v1, "echo:v1")

ray.serve.link("my_endpoint", "echo:v1")

# Make sure the Docker container doesn't exit
while True:
    time.sleep(2)

Without the last part:

# Make sure the Docker container doesn't exit
while True:
    time.sleep(2)

The Docker container will immediately exit.

score 0 · Answer 1 · answered Feb 07 '22 at 14:28

From the ray documentation as well (https://docs.ray.io/en/latest/serve/deployment.html), you can start a server beforehand with ray start --head and connect to the server with ray.init(address="auto"):

ray start --head

import ray
from ray import serve

# This will connect to the running Ray cluster.
ray.init(address="auto", namespace="serve")

@serve.deployment
def my_func(request):
  return "hello"

my_func.deploy()

Running ray serve in a docker container in foreground, not daemon mode

1 Answers1