Main Issue: Every time I try to debug my quantized model, my jupter kernel dies and is restarted. Want to know what to fix, but more importantly, how to look for what to fix the next time this happens.
What I've tried:
- Not sure about behavior of debugger and need for GPU, but I thought maybe it was running out of GPU memory after I had trained the model (because in Jupyter notebook it has a habit of not releasing the memory?). So I killed the kernel, loaded model from save file, then tried to run debugger. No luck.
- I think my representative data is setup correctly. Shape is (1,52) which matches model input. Reduce samples to 50 after thinking it was a RAM issue, but my system monitor says RAM is NOWHERE near exhausted, so it didn't work.
- Tutorial works for me too, so it's something peculiar to my setup.
Background + Code:
Custom model and data, but my quantization debugging follows this tutorial.
Representative data setup:
def representative_dataset(df):
def _data_gen():
df_to_process = df.copy()
# Only took 50 samples because at point I wasn't sure if RAM was an issue. Really small though
X = df_to_process.iloc[np.random.choice(len(X_test), 50)].astype('float32')
dataset = tf.data.Dataset.from_tensor_slices((tf.cast(X.values, tf.float32)))
for data in dataset.batch(1):
yield [data]
return _data_gen
Tflite converter and output:
# quant_aware_model is fully trained model
converter = tf.lite.TFLiteConverter.from_keras_model(quant_aware_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset(X_test)
debugger = tf.lite.experimental.QuantizationDebugger(converter=converter, debug_dataset=representative_dataset(X_test))
tflite_model = converter.convert()
WARNING:absl:Found untraced functions such as dense_21_layer_call_fn, dense_21_layer_call_and_return_conditional_losses, dense_22_layer_call_fn, dense_22_layer_call_and_return_conditional_losses, dense_23_layer_call_fn while saving (showing 5 of 14). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: /tmp/tmpim6lhlmi/assets
INFO:tensorflow:Assets written to: /tmp/tmpim6lhlmi/assets
/home/smallrig/Documents/Dev/Projects/DataScience/ComputerVision/movement_analyzer/venv/lib/python3.7/site-packages/tensorflow/lite/python/convert.py:746: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
warnings.warn("Statistics for quantized inputs were expected, but not "
2022-02-28 14:58:12.672894: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:357] Ignored output_format.
2022-02-28 14:58:12.672916: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:360] Ignored drop_control_dependency.
2022-02-28 14:58:12.673089: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: /tmp/tmpim6lhlmi
2022-02-28 14:58:12.676308: I tensorflow/cc/saved_model/reader.cc:78] Reading meta graph with tags { serve }
2022-02-28 14:58:12.676320: I tensorflow/cc/saved_model/reader.cc:119] Reading SavedModel debug info (if present) from: /tmp/tmpim6lhlmi
2022-02-28 14:58:12.686773: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2022-02-28 14:58:12.749001: I tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: /tmp/tmpim6lhlmi
2022-02-28 14:58:12.772340: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 99251 microseconds.
fully_quantize: 0, inference_type: 6, input_inference_type: 0, output_inference_type: 0
WARNING:absl:Found untraced functions such as dense_21_layer_call_fn, dense_21_layer_call_and_return_conditional_losses, dense_22_layer_call_fn, dense_22_layer_call_and_return_conditional_losses, dense_23_layer_call_fn while saving (showing 5 of 14). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: /tmp/tmppkrp6rod/assets
INFO:tensorflow:Assets written to: /tmp/tmppkrp6rod/assets
/home/smallrig/Documents/Dev/Projects/DataScience/ComputerVision/movement_analyzer/venv/lib/python3.7/site-packages/tensorflow/lite/python/convert.py:746: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
warnings.warn("Statistics for quantized inputs were expected, but not "
2022-02-28 14:58:15.810206: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:357] Ignored output_format.
2022-02-28 14:58:15.810227: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:360] Ignored drop_control_dependency.
2022-02-28 14:58:15.810387: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: /tmp/tmppkrp6rod
2022-02-28 14:58:15.817820: I tensorflow/cc/saved_model/reader.cc:78] Reading meta graph with tags { serve }
2022-02-28 14:58:15.817849: I tensorflow/cc/saved_model/reader.cc:119] Reading SavedModel debug info (if present) from: /tmp/tmppkrp6rod
2022-02-28 14:58:15.834057: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2022-02-28 14:58:15.904993: I tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: /tmp/tmppkrp6rod
2022-02-28 14:58:15.932282: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 121895 microseconds.
and finally, debugger.run() kills my kernel
