I'm running Python 3.9.7 and TensorFlow 2.5 on my MacBook Pro with M1 Max chip, in a virtual environment managed through MiniForge. (Setup due to Jeff Heaton).
I have a custom loss function in the model. This code runs fine on my other machine, which is Ubuntu-based Pop!OS.
When I run model.fit from Keras, I occasionally get the following errors. Sometimes it runs just fine. Any idea what's causing it, why it only happens sometimes, and how to fix it?
In addition, it only produces this error from my custom loss function. Not from any Keras-native.
Thanks!
:5:10: error: invalid length (0) for size 20 | 0.00/1.00 [00:00<?, ?batch/s]
-:5:10: error: 'mps.slice' op failed to fold or infer return types
-:5:10: note: see current operation: %2 = "mps.slice"(%arg0) {axis = 0 : si32, length = 999 : i32, start = 1 : i32} : (tensor<20xf32>) -> tensor<*xf32>
/System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/MetalPerformanceShadersGraph-2.2.1/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:1179: failed assertion `Error: MLIR pass manager failed'
model.fit(tf.convert_to_tensor(Σ),tf.zeros((T,output)),batch_size=T,epochs=1,verbose=0,callbacks=[TqdmCallback()])#,tbc])
zsh: abort python3
(tensorflow) kpmott@kevins-mbp-7 tf.olg % model.fit(tf.convert_to_tensor(Σ),tf.zeros((T,output)),batch_size=T,epochs=1,verbose=0,callbacks=[TqdmCallback()])#,tbc])
zsh: parse error near `)'
(tensorflow) kpmott@kevins-mbp-7 tf.olg % /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
model.fit(tf.convert_to_tensor(Σ),tf.zeros((T,output)),batch_size=T,epochs=1,verbose=0,callbacks=[TqdmCallback()])#,tbc])
zsh: parse error near `)'
