debugging python autosklearn on wsl 2 ubuntu with vs code

Question

I have a python script using autosklearn. I am running this under wsl ubuntu 1804, because the package didn't intall in windows.

I've been running jupyter notebook to create the script initially. It runs fine in the browser, but in order to go further, I prefer to use vs code. I installed vs code server, and can debug the script in vscode. Initially pylint didn't recognise my autosklearn import. Then I installed the Microsoft python extension on ubuntu, and it was happy.

I should point out I was previously running jupyter notebook with associated packages from an isolated environment. I started vscode from within that environment.

When I debug the script, everything is fine until I get to the line that does the fitting

automl = autosklearn.regression.AutoSklearnRegressor(
                        time_left_for_this_task=360,
                        per_run_time_limit=30,
                        tmp_folder='./autosklearn_regression_example_tmp',
                        output_folder='./autosklearn_regression_example_out',
                        )
automl.fit(X_train, y_train.values.ravel(),
       dataset_name='mytest',
       feat_type=feature_types)

I get this error:

E00005.591: Exception escaped from start_client

        Traceback (most recent call last):
          File "/home/ubx/.vscode-server/extensions/ms-python.python-2019.11.49689/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
            return f(*args, **kwargs)
          File "/home/ubx/.vscode-server/extensions/ms-python.python-2019.11.49689/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
            sock, start_session = daemon.start_client((host, port))
          File "/home/ubx/.vscode-server/extensions/ms-python.python-2019.11.49689/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
            with self.started():
          File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
            return next(self.gen)
          File "/home/ubx/.vscode-server/extensions/ms-python.python-2019.11.49689/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
            self.start()
          File "/home/ubx/.vscode-server/extensions/ms-python.python-2019.11.49689/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
            raise RuntimeError('already started')
        RuntimeError: already started


Traceback (most recent call last):

Terminated

I've got a feeling that the problem is that microsoft's version of python (which was installed as an extension for vscode) is incompatible with the package. The package still runs fine from within ubuntu jupyter notebook.

But the error is very odd. I would expect a different error for package incompatibility.

Any suggestions much appreciated.

Martin

FYI there is no "microsoft's version of python". Microsoft has no Python distribution and the Python extension for VS Code never installs Python. — Brett Cannon, Nov 20 '19 at 19:54

score 0 · Answer 1 · answered Nov 27 '19 at 01:23

The error stack that you're seeing is indicative of the library attempting to fork the process, e.g. because it's using the standard multiprocessing module internally. The version of ptvsd that you're running it on does not support forking.

If it does indeed use multiprocessing, you should be able to explicitly tell it to use spawn instead of fork in your main script:

import multiprocessing

if __name__ == "__main__":
    multiprocessing.set_start_method("spawn")

The part that checks __name__ is important to avoid child processes from running the same line. A more detailed explanation of various start methods and their gotchas can be found in Python documentation. Note that as of Python 3.8, "spawn" is the default on Windows and MacOS, and "fork" is outright unavailable on the former, and considered broken on the latter - so code that wants to run on all three, and expects consistent behavior, should use "spawn".

The upcoming major revision of ptvsd fully supports forking, but it hasn't been released yet. You can track the progress here.

debugging python autosklearn on wsl 2 ubuntu with vs code

1 Answers1