0

I am trying to deploy a dummy model to AWS Sagemaker using Databricks and MLflow.

According to this documentation, it builds a new MLflow Sagemaker image, assigns it a name, and push to ECR.

However, when I run the following lines of code in a Databricks notebook:

%sh
mlflow sagemaker build-and-push-container

I get the following error:

/databricks/python3/lib/python3.9/site-packages/click/core.py:2309: UserWarning: Virtualenv support is still experimental and may be changed in a future release without warning.
  value = self.callback(ctx, self, value)
2023/06/12 16:22:04 INFO mlflow.models.docker_utils: Building docker image with name mlflow-pyfunc
Traceback (most recent call last):
  File "/databricks/python3/bin/mlflow", line 8, in <module>
    sys.exit(cli())
  File "/databricks/python3/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/databricks/python3/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/databricks/python3/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/databricks/python3/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/databricks/python3/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/databricks/python3/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/databricks/python3/lib/python3.9/site-packages/mlflow/sagemaker/cli.py", line 605, in build_and_push_container
    mlflow.models.docker_utils._build_image(
  File "/databricks/python3/lib/python3.9/site-packages/mlflow/models/docker_utils.py", line 224, in _build_image
    _build_image_from_context(context_dir=cwd, image_name=image_name)
  File "/databricks/python3/lib/python3.9/site-packages/mlflow/models/docker_utils.py", line 228, in _build_image_from_context
    import docker
ModuleNotFoundError: No module named 'docker'

I have tried installing docker by running %pip install docker and I have tried Databricks' built-in package manager to install docker (using the UI) but the problem still persists.

Could I please get some assistance trying to figure this out? Thanks in advance!

nikhil
  • 1
  • 1
  • Hi, I see you're new to SO. If you feel an answer solved the problem, please mark it as 'accepted' by clicking the green check mark. This helps keep the focus on older SO which still don't have answers. – fucalost Jun 25 '23 at 16:51

1 Answers1

0

You'll need to attempt this on a non-managed instance. Using %pip install docker will install the package in whichever virtual environment is connected to your notebook. Also, that command installs a Python wrapper for docker, not the engine itself.

Quick fix: provision an EC2 instance, install MLFlow and docker, then retry the process.

fucalost
  • 313
  • 1
  • 8