I am getting this error when trying to load the saved .h5 model in tensorflow:
load_model("path_to_model.h5", custom_objects={"TFBertModel": TFBertModel, "AdamWeightDecay": AdamWeightDecay})
ValueError: Cannot assign value to variable
' tf_bert_model/bert/embeddings/word_embeddings/weight:0': Shape mismatch.The variable shape (30523, 768), and the assigned value shape (2, 768) are incompatible.`
Steps to reproduce the error:
- load the pretrained bert-base-uncased model and tokenizer from the transformers library
- add a new token to the tokenizer and resize token embeddings
- finetune the model
- save the model
- load the model
Everything works fine if I don't add new tokens. The problem arises only when I add a new token:
tokenizer.add_tokens(['new_token'])
bert.resize_token_embeddings(len(tokenizer))