I am currently trying to import a pretrained BERT model for Text classification. After days of searching the internet I've not been able to find a fix. Using Python veriosn 3.8.2 and tensorflow version 2.4.1
The pretrained model can be found here: https://github.com/botxo/nordic_bert which contrains the following file: bert_config.json, bert_model.ckpt.data-00000-of-00001, bert_model.ckpt.index, bert_model.ckpt.meta, vocab.txt
Here is some code:
import tensorflow as tf
with tf.compat.v1.Session() as sess:
saver = tf.compat.v1.train.import_meta_graph('insert_path/bert_model.ckpt.meta'), clear_devices=True)
saver.restore(sess, 'insert_path/bert_model.ckpt')
checkpoint = tf.train.Checkpoint(encoder=bert_encoder)
checkpoint.read('inser_path/bert_model.ckpt')).assert_consumed()
What i'm trying to do is import the model/checkpoint and finetune on custom data to later use for sentence classification. The problem occurs when doing the saver.restore where I recieve the following error:
Traceback (most recent call last):
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 1375, in _do_call
return fn(*args)
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 1358, in _run_fn
self._extend_graph()
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 1398, in _extend_graph
tf_session.ExtendSession(self._session)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'InfeedEnqueueTuple' used by {{node input_pipeline_task0/while/InfeedQueue/enqueue/0}} with these attrs: [_class=["loc:@input_pipeline_task0/while/IteratorGetNext"], device_ordinal=0, shapes=[[128,128], [128,128], [128,20], [128,20], [128,20], [128,1], [128,128]], dtypes=[DT_INT32, DT_INT32, DT_INT32, DT_INT32, DT_FLOAT, DT_INT32, DT_INT32], layouts=[]]
Registered devices: [CPU]
Registered kernels:
<no registered kernels>
[[input_pipeline_task0/while/InfeedQueue/enqueue/0]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/training/saver.py", line 1297, in restore
sess.run(self.saver_def.restore_op_name,
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 967, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 1190, in _run
results = self._do_run(handle, final_targets, final_fetches,
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 1368, in _do_run
return self._do_call(_run_fn, feeds, fetches, targets, options,
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/client/session.py", line 1394, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'InfeedEnqueueTuple' used by node input_pipeline_task0/while/InfeedQueue/enqueue/0 (defined at <input>:109) with these attrs: [_class=["loc:@input_pipeline_task0/while/IteratorGetNext"], device_ordinal=0, shapes=[[128,128], [128,128], [128,20], [128,20], [128,20], [128,1], [128,128]], dtypes=[DT_INT32, DT_INT32, DT_INT32, DT_INT32, DT_FLOAT, DT_INT32, DT_INT32], layouts=[]]
Registered devices: [CPU]
Registered kernels:
<no registered kernels>
[[input_pipeline_task0/while/InfeedQueue/enqueue/0]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 5, in <module>
File "/Users/ah/Library/Python/3.8/lib/python/site-packages/tensorflow/python/training/saver.py", line 1333, in restore
raise _wrap_restore_error_with_msg(
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
No OpKernel was registered to support Op 'InfeedEnqueueTuple' used by node input_pipeline_task0/while/InfeedQueue/enqueue/0 (defined at <input>:109) with these attrs: [_class=["loc:@input_pipeline_task0/while/IteratorGetNext"], device_ordinal=0, shapes=[[128,128], [128,128], [128,20], [128,20], [128,20], [128,1], [128,128]], dtypes=[DT_INT32, DT_INT32, DT_INT32, DT_INT32, DT_FLOAT, DT_INT32, DT_INT32], layouts=[]]
Registered devices: [CPU]
Registered kernels:
<no registered kernels>
[[input_pipeline_task0/while/InfeedQueue/enqueue/0]]
I'm using a Macbook pro 16" 2019 with AMD Radeon Pro 5300M and 8-core Intel i9 processor if that matters.
