0

I used this code to load weights

from transformers import DebertaTokenizer, DebertaModel
import torch

tokenizer = DebertaTokenizer.from_pretrained('microsoft/deberta-base')
model = DebertaModel.from_pretrained('microsoft/deberta-base')

after that i want to optimize and use loss function using compile function

model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=tf.metrics.SparseCategoricalAccuracy(),
)


I got this error AttributeError: 'DebertaModel' object has no attribute 'compile'

Shorouk Adel
  • 127
  • 3
  • 20

2 Answers2

2

The transformers library offers a tensorflow-based model TFDebertaModel. Replace DebertaModel with TFDebertaModel and the execution of compile() works.

I changed your snippet to the following and compile works. Try this:

from transformers import DebertaTokenizer, TFDebertaModel

Tested with transformers version: 4.19.2

https://huggingface.co/docs/transformers/v4.19.2/en/model_doc/deberta#transformers.TFDebertaModel

rbi
  • 356
  • 1
  • 4
1

the only way to work on it using pytorch library

Shorouk Adel
  • 127
  • 3
  • 20