How to get Pre-trained XLNET Sentence embeddings?

Question

I want to get XLNET Pre-trained Sentence Embeddings of any given sentence. Please provide a code snippet to get the embeddings

Answer 1

You can refer to the Hugging Face XLNet model usage example. For more detail of how to use transformers, please refer to the Hugging Face official course.

from transformers import XLNetTokenizer, XLNetModel
import torch

tokenizer = XLNetTokenizer.from_pretrained("xlnet-base-cased")
model = XLNetModel.from_pretrained("xlnet-base-cased")

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state