how to use deberta model from hugging face and use .compile() and . summary() with it

Question

I used this code to load weights

from transformers import DebertaTokenizer, DebertaModel
import torch

tokenizer = DebertaTokenizer.from_pretrained('microsoft/deberta-base')
model = DebertaModel.from_pretrained('microsoft/deberta-base')

after that i want to optimize and use loss function using compile function

model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=tf.metrics.SparseCategoricalAccuracy(),
)

I got this error AttributeError: 'DebertaModel' object has no attribute 'compile'

Tensorflow based models has a compile and summary function. But this model pytorch based — Birol Kuyumcu, Jul 14 '21 at 13:55
thank you but how to use compile with it cuz i use keras in my code — Shorouk Adel, Jul 14 '21 at 14:01

score 2 · Answer 1 · answered Jun 07 '22 at 12:09

The transformers library offers a tensorflow-based model TFDebertaModel. Replace DebertaModel with TFDebertaModel and the execution of compile() works.

I changed your snippet to the following and compile works. Try this:

from transformers import DebertaTokenizer, TFDebertaModel

Tested with transformers version: 4.19.2

https://huggingface.co/docs/transformers/v4.19.2/en/model_doc/deberta#transformers.TFDebertaModel

score 1 · Answer 2 · answered Aug 25 '21 at 14:49

1

the only way to work on it using pytorch library

answered Aug 25 '21 at 14:49

Shorouk Adel

127
3
20

how to use deberta model from hugging face and use .compile() and . summary() with it

2 Answers2