1

I'm attempting to read a csv file using modin and it results in the following error. this issue seems to happen on all dataframe operations:

RayWorkerError: The worker died unexpectedly while executing this task.

Python 3.7.3 Pandas 0.24.2 Modin 0.5.4 Ray 0.7.1

import modin.pandas as pd
import numpy as np    

frame_data = np.random.randint(0, 100, size=(2**10, 2**8))
pd.DataFrame(frame_data).to_csv('frame_data.csv')
pd.read_csv('frame_data.csv').head()
DACW
  • 2,601
  • 2
  • 18
  • 16
  • This issue is tracked in the ray project as a "stability blocker" as of 07-August-2019 in the following location : https://github.com/ray-project/ray/issues/5316 – DACW Aug 08 '19 at 19:37
  • 1
    The comment at https://github.com/ray-project/ray/issues/5316#issuecomment-519367386 suggests that doing `pip uninstall typing` may fix the issue. There appears to be a bug with the `typing` module in Python 3.7. – Robert Nishihara Aug 17 '19 at 20:02
  • 1
    Yes, `pip uninstall typing` does work fine. This raises the question of how well integration tested before releasing in PyPI. Uninstalling packages like this in a production environment without considering the dependencies for other packages dosent seem ideal. – DACW Aug 21 '19 at 22:07
  • 1
    It's definitely not ideal. I *think* that it's an issue with the `typing` module that needs to be fixed. But maybe there's something that can be done on the Ray side to work around it. – Robert Nishihara Aug 22 '19 at 05:45

1 Answers1

1

OP confirmed that the reason for the failure was the presence of the typing package, and that uninstalling typing fished the issue. That was a temporary fix for the issue tracked on Ray here. That issue was closed once Modin fixed the order of imports for the typing library. The latest version of Modin (0.12.0) should not have that problem.

Mahesh Vashishtha
  • 166
  • 5