Stack Overflow
Stack Overflow

Questions tagged [distilbert]

37 questions
1
vote
0 answers

DistilBERT Prediction Output - "TypeError: only size-1 arrays can be converted to Python scalars"

I am trying to apply a DistilBERT model to create a prediction, whether a sentence is a Claim, Premise or Non-Argumentative (3 Outputs) However when I apply the model and want to create a prediction with the following code: def…
Philipp
  • 11
  • 1
1
vote
1 answer

distillbert ktrain 'too many values to unpack'

I am trying to run DistilBert with ktrain in Colab but I am getting "error too many values to unpack". I am trying to perform toxic comment classification, I uploaded 'train.csv' from CivilComments, I am able to run BERT but not…
semper pi
  • 11
  • 2
1
vote
1 answer

fine-tuning huggingface DistilBERT for multi-class classification on custom dataset yields weird output shape on prediction

I'm trying to fine-tune huggingface's implementation of distilbert for multi-class classification (100 classes) on a custom dataset following the tutorial at https://huggingface.co/transformers/custom_datasets.html. I'm doing so using Tensorflow,…
roberta
  • 131
  • 1
  • 12
1
vote
1 answer

Incompatible shapes: [11,768] vs. [1,5,768] - Inference in production with a huggingface saved model

I have saved a pre-trained version of distilbert, distilbert-base-uncased-finetuned-sst-2-english, from huggingface models, and i am attempting to serve it via Tensorflow Serve and make predictions. All is being tested currently in Colab at the…
JSS
  • 174
  • 9
1
vote
1 answer

Issue when preprocessing text with Ktrain and DistilBERT

Following the example notebook here: https://github.com/amaiya/ktrain/blob/master/examples/text/20newsgroup-distilbert.ipynb At STEP 1: Preprocess Data, I run into the errors listed below. When I do exactly the same in a Colab notebook, it works.…
T_202
  • 21
  • 6
0
votes
0 answers

(tflite_flutter) tflite model (text classification) giving the same result

I'm using tensorflow (Distilbert) to classify text. I have used tflite_flutter package to run text classification using Distilbert to classify topic from text. The training model is shown below using: dbert_tokenizer =…
jerry budiman
  • 1
0
votes
1 answer

Fine tuning error : Expected input batch_size (1) to match target batch_size (64)

I am new to pytorch. I'm fine tuning a Bert model to do a text binary-classification. Once I run my code, I get this following error : Expected input batch_size (1) to match target batch_size (64). Here's my model structure : from torch import nn #…
Akram H
  • 33
  • 4
0
votes
0 answers

SentenceTransformer ('distilbert-base-nli-mean-tokens') is very slow

I am trying to learn the use of BERT. Here is the code: from sklearn.datasets import fetch_20newsgroups data = fetch_20newsgroups(subset='all')['data'] from sentence_transformers import SentenceTransformer model =…
Toly
  • 2,981
  • 8
  • 25
  • 35
0
votes
2 answers

This code always predicts a "period" as the next text sequence

I am trying to learn how to use the transformers library to make predictions on the next word given a sentence. My code always predicts a "period" as the next token. Can someone help me see what I am doing wrong? import torch from transformers…
steve landiss
  • 1,833
  • 3
  • 19
  • 30
0
votes
0 answers

Can't find Huggingface npm packages for node.js?

I am trying to execute a code in node.js using DistilBERT and need to install the below dependencies. They don't seem to work. Anyone knows? npm install @huggingface/transformers npm install @huggingface/tokenizers
tar111
  • 1
  • 1
0
votes
0 answers

BERT vs distilBERT: reduce the number of layers

I am exploring BERT model and its distiled version - distilBERT. I am reading to part 3 of DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter paper and know that the number of layer of distiBERT is reduced by factor of…
PhuongBui712
  • 11
  • 2
0
votes
0 answers

How to save tokenizer DistilBertTokenizer after reading my x train values

I am using Transformers and DistilBert for text classification. My dataset is 700000 rows and It is a bit heavy. I am running my code on Google colab. I used this code before building my model. X = dfreadtrain['review_text'].values y =…
GSandro_Strongs
  • 773
  • 3
  • 11
  • 24
0
votes
0 answers

Distilbert Autotokenizer for custom data:: TypeError: PreTokenizedEncodeInput must be Union[PreTokenizedInputSequence, T

I am new to distillbert and want to use it for token classification. I have my own dataset and own class labels. Used the following function to tokenize. tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased") label_all_tokens=True def…
sak
  • 59
  • 4
0
votes
0 answers

Twitter Sentiment Analysis : TypeError: dropout(): argument 'input' (position 1) must be Tensor, not tuple in DistilBERT using huggingface library

The following is my Sentiment Analyser: from transformers import DistilBertTokenizer, DistilBertModel PRE_TRAINED_MODEL_NAME = 'distilbert-base-cased' db_model = DistilBertModel.from_pretrained(PRE_TRAINED_MODEL_NAME, return_dict = False) tokenizer…
wamika
  • 21
  • 1
  • 8
0
votes
1 answer

What does pre_classifier in distilbert do?

I was reading huggingface's DistilBertForSequenceClassification implementation code and noticed that they create a classifier and a pre_classifier when initiating the object. Later in the forward method they send the pooled output to the…
m.m
  • 125
  • 7
1
2
3