python ray not completing all remote calls

Question

so i have a simple a script

import ray
import requests as r

ray.init()

@ray.remote
def f(i):
    print(i)
    r.get("http://127.0.0.1:5000/" + str(i))
    return i

if __name__ == "__main__":
    for k in range(1000):
        f.remote(k)

when i run this script only 250 remote calls happens and the program terminates instead of executing all the 1000 remote calls.

i really have no clue even after viewing the ray browser dashboard

when i set local_mode=True the program complete all the 1000 calls

score 2 · Accepted Answer · answered Sep 20 '20 at 17:19

f.remote(k) starts executing the remote calls, but doesn't wait for the calls to finish. Therefore the program finishes and shuts down before the calls finish.

You should use ray.get to make sure the calls finish.

refs = []
for k in range(1000):
    refs.append(f.remote(k)
ray.get(refs)

See this answer for more details.

python ray not completing all remote calls

1 Answers1