python import fails in Visual Studio Code devcontainer but works in local docker container

Question

I'm trying to develop a python script using a devcontainer. I can run the script in a local docker container but the import fails when running in the devcontainer. The devcontainer error appears to be a problem with _cffi_backend.

I'm using python 3.11.

The code.

import boto3

client = boto3.client('s3')

print(f"client is not none: {client is not None}")

I'm expecting the results from the devcontainer to match the results from the local container.

Startup the docker container and install the boto3 dependency.

docker run --rm -it -v D:\MDVIP\Dummy\BotoTest:/tmp --entrypoint bash jupyter/pyspark-notebook
pip install boto3

Running test.py in the local container with:

python /tmp/test.py

Returns what I expected.

client is not none: True

Launching test.py in the Visual Studio Code devcontainer using the following launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true
        }
    ]
}

returns the following error.

Exception has occurred: PanicException
Python API call failed
  File "/workspaces/BotoTest/test.py", line 1, in <module>
    import boto3
pyo3_runtime.PanicException: Python API call failed

The terminal returns the following. The elided lines are repeated patterns I removed.

cd /workspaces/BotoTest ; /usr/bin/env /bin/python3 /home/jovyan/.vscode-server/extensions/ms-python.python-2023.10.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 36813 -- /workspaces/BotoTest/test.py 
ModuleNotFoundError: No module named '_cffi_backend'
thread '<unnamed>' panicked at 'Python API call failed', /home/conda/feedstock_root/build_artifacts/cryptography-split_1685659433210/_build_env/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.18.3/src/err/mod.rs:790:5
stack backtrace:
   0: std::panicking::begin_panic
   1: pyo3::err::panic_after_error
   2: cryptography_cffi::create_module
   3: cryptography_rust::_rust
   4: pyo3::impl_::pymodule::ModuleDef::make_module
   5: PyInit__rust
   6: <unknown>
   7: <unknown>
   8: _PyEval_EvalFrameDefault
   9: _PyFunction_Vectorcall
  10: <unknown>
  11: _PyEval_EvalFrameDefault
  12: _PyFunction_Vectorcall
  13: <unknown>
  ...
  26: _PyObject_CallMethodIdObjArgs
  27: PyImport_ImportModuleLevelObject
  28: _PyEval_EvalFrameDefault
  29: <unknown>
  30: PyEval_EvalCode
  31: <unknown>
  32: <unknown>
  33: _PyEval_EvalFrameDefault
  34: _PyFunction_Vectorcall
  35: <unknown>
  36: _PyEval_EvalFrameDefault
  37: _PyFunction_Vectorcall
  38: <unknown>
  ...
  48: _PyObject_CallMethodIdObjArgs
  49: PyImport_ImportModuleLevelObject
  50: _PyEval_EvalFrameDefault
  51: <unknown>
  52: PyEval_EvalCode
  53: <unknown>
  54: <unknown>
  55: _PyEval_EvalFrameDefault
  56: _PyFunction_Vectorcall
  57: <unknown>
  58: _PyEval_EvalFrameDefault
  59: _PyFunction_Vectorcall
  60: <unknown>
  ...
  89: _PyEval_EvalFrameDefault
  90: _PyFunction_Vectorcall
  91: <unknown>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

The Docker file.

FROM jupyter/pyspark-notebook

RUN pip install boto3

The devcontainer.

{
    "name": "Python 3",
    "build": {
        "dockerfile": "Dockerfile",
        "context": ".."
    },

    "containerEnv": {
        "PYTHONPATH": "/opt/conda/lib/python3.11/site-packages"
    },

    "customizations": {
        "vscode": {
            // Set *default* container specific settings.json values on container create.
            "settings": {
                "python.pythonPath": "/opt/conda/lib/python3.11/site-packages"
            },
            "extensions": [
                "ms-python.python",
                "ms-python.vscode-pylance"
            ]
        }

    },
}