Questions tagged [apache-arrow-plasma]

I am trying to load and process large files using ray. I am using ray for the purpose of multiprocessing the files and improving the speed of the solution. I keep running into this pyarrow error: pyarrow.lib.ArrowInvalid: Maximum size exceeded…

I'm relatively new to plasma. Want to ask after connecting to the client using Python API pyarrow.plasma, is there any API to find the remaining available memory of the current plasma object store? Seems using client.list() can get all objects and…

I am using ray.put(large_2d_array) to store a large numpy 2D boolean array and then in the worker process I take a column from this shared numpy array and pass it to a cython function and then create a flat buffer view via cdef cnp.npy_bool view =…