1

I am getting this error when trying to load the saved .h5 model in tensorflow:

load_model("path_to_model.h5", custom_objects={"TFBertModel": TFBertModel, "AdamWeightDecay": AdamWeightDecay})

ValueError: Cannot assign value to variable 
' tf_bert_model/bert/embeddings/word_embeddings/weight:0': Shape mismatch.The variable shape (30523, 768), and the assigned value shape (2, 768) are incompatible.`

Steps to reproduce the error:

  • load the pretrained bert-base-uncased model and tokenizer from the transformers library
  • add a new token to the tokenizer and resize token embeddings
  • finetune the model
  • save the model
  • load the model

Everything works fine if I don't add new tokens. The problem arises only when I add a new token:

tokenizer.add_tokens(['new_token'])
bert.resize_token_embeddings(len(tokenizer))
richard
  • 11
  • 1

0 Answers0