1

I'm using Spacy-Transformers to build some NLP models.

The Spacy-Transformers docs say:

spacy-transformers

spaCy pipelines for pretrained BERT, XLNet and GPT-2

The sample code on that page shows:

import spacy

nlp = spacy.load("en_core_web_trf")
doc = nlp("Apple shares rose on the news. Apple pie is delicious.")

Based on what I've learned from this video,"en_core_web_trf" appears to be the spacy.load() package to use a BERT model. I've searched the Spacy-Transformers docs and haven't yet seen an equivalent package, to access GPT-2. Is there a specific spacy.load() package, to load in order to use a GPT-2 model?

VikR
  • 4,818
  • 8
  • 51
  • 96

1 Answers1

1

The en_core_web_trf uses a specific Transformers model, but you can specify arbitrary ones using the TransformerModel wrapper class from spacy-transformers. See the docs for that. An example config:

[model]
@architectures = "spacy-transformers.TransformerModel.v1"
name = "roberta-base" # this can be the name of any hub model
tokenizer_config = {"use_fast": true}
polm23
  • 14,456
  • 7
  • 35
  • 59
  • Could you please post a link to the list of available models? I found this, but it doesn't seem to include GPT-2, or roberta-base: https://huggingface.co/transformers/search.html?q=roberta-base&check_keywords=yes&area=default – VikR Aug 28 '21 at 08:25
  • 1
    Model names come from here. Not sure how to filter for the compatible ones, but it's a lot of them. https://huggingface.co/models – polm23 Aug 28 '21 at 12:33