0

I am doing a research on pre-trained LMs, specifically the following LMs:

  • BERT
  • ALBERT
  • RoBERTa
  • XLNet
  • DistilBERT
  • BigBird
  • ConvBERT

I am looking for information to compare these LMs like: number of parameters, layers, data on which they were pre-trained...

In other words, I want to extend the following table to the other LMs:

enter image description here

But I can't find information online! Can you please help me?

Thanks.

Othman El houfi
  • 53
  • 3
  • 9
  • You need to read the documentation for each of these models, possibly even the research paper associated with it. – Erwan Jun 10 '22 at 10:15
  • Thanks @Erwan actually that's exactly what I am doing, I just thought maybe I can use some help tracking these informations to finish fast. – Othman El houfi Jun 10 '22 at 12:05

0 Answers0