VScode kernel crash when trying to import and predict with 'tensorflow.keras.applications'

Question

I try to run a Jupyter notebook on my machine. And here is what I want to do:

1. load an image
2. load a pre-train model from 'tensorflow.keras.applications'
3. make prediction

Here is the sample code

from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input, decode_predictions

model = ResNet50(weights='imagenet', include_top=True)
img = tf.keras.preprocessing.image.load_img(imgfile,
                                            grayscale=False, 
                                            color_mode='rgb',
                                            target_size=(224,224),)

img = tf.keras.preprocessing.image.img_to_array(img)
img = preprocess_input(img)
img = np.reshape(img,(-1,224,224,3))

preds = model.predict(img) # crash at this line

Then the kernel crash with message below:

warn 15:23:44.556: StdErr from Kernel Process 2022-05-22 15:23:44.55 6754: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8400 error 15:23:45.519: Disposing session as kernel process died ExitCode: 3221226505, Reason: C:\XXXXX\Python39\site-packages\traitlets\traitlets.py:2202: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn( 
C:\XXXXX\Python39\site-packages\traitlets\traitlets.py:2157: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use '7176e2c9-27f6-42c2-8ae4-00b7ea3c655d' instead of 'b"7176e2c9-27f6-42c2-8ae4-00b7ea3c655d"'.
  warn(
2022-05-22 15:23:36.484356: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-22 15:23:37.856705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3969 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
2022-05-22 15:23:44.556754: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8400

info 15:23:45.520: Dispose Kernel process
info 15:23:45.520: kill daemon
error 15:23:45.520: Raw kernel process exited code: 3221226505
error 15:23:45.522: Error in waiting for cell to complete [Error: Canceled future for execute_request message before replies were done
    at t.KernelShellFutureHandler.dispose (c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:1327723)
    at c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:1346775
    at Map.forEach (<anonymous>)
    at v._clearKernelState (c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:1346760)
    at v.dispose (c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:1340242)
    at c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:539831
    at t.swallowExceptions (c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:924542)
    at u.dispose (c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:539809)
    at t.RawSession.dispose (c:\XXXXX\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001311010\out\extension.node.js:2:543488)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)]
warn 15:23:45.523: Cell completed with errors {
  message: 'Canceled future for execute_request message before replies were done'
}
info 15:23:45.525: Cancel all remaining cells true || Error || undefined
info 15:23:45.525: Cancel pending cells
info 15:23:45.525: Cell 5 executed with state Error

---

Environment

• Win10
• Python 3.9.6
• VScode 1.67.0
  • Jupyter extension v2022.5.1001311010
  • Python extension v2022.6.2
• CUDA v11.2 & v11.6
  • not sure which version is used by tensorflow

---

I search some articles, some say it might be the incomplete installation of GPU development settings, some say it is a bug from VScode jupyter extension.

Unfortunately, I have no luck to solve my issue.

Any suggestion is appreciated.

Thanks!

Answer 1

Providing the solution here (Answer Section), even though it is present in the Comment Section for the benefit of the community.

This issue is due to version incompatibility. As per the tested build configurations, Tensorflow 2.9.0 is compatible with CUDA 11.2 and cuDNN 8.1. Downgrading it to cuDNN 8.1 and using CUDA 11.2 will help.