The huggingface tag can be used for all libraries made by Hugging Face. Please ALWAYS use the more specific tags; huggingface-transformers, huggingface-tokenizers, huggingface-datasets if your question concerns one of those libraries.
Questions tagged [huggingface]
606 questions
1
vote
1 answer
Sentence embeddings from LLAMA 2 Huggingface opensource
Could anyone let me know if there is any way of getting sentence embeddings from meta-llama/Llama-2-13b-chat-hf from huggingface?
Model link: https://huggingface.co/meta-llama/Llama-2-13b-chat-hf
I tried using transfomer.Automodel module from…
Mukesh Reddy
- 11
- 1
1
vote
1 answer
Validation and Training Loss when using HuggingFace
I do not seem to find an explanation on how the validation and training losses are calculated when we finetune a model using the huggingFace trainer. Does anyone know here to find this information?
tt40kiwi
- 361
- 1
- 8
1
vote
1 answer
How to download data from hugging face that is visible on the data viewer but the files are not available?
I can see them (data set link hf: https://huggingface.co/datasets/EleutherAI/pile/) :
but no matter how I change the download url I can't get the data. Files are not there and their script doesn't work.
Anyone know how to get the splits and know…
Charlie Parker
- 5,884
- 57
- 198
- 323
1
vote
1 answer
Unable to run a model using HuggingFace Inference Endpoints
I am able to make successful requests using the free endpoint, but when using Inference Endpoints, I get 404 response. Here is the relevant piece of code:
mode = 'paid' # works if 'free'
model_id =…
AlwaysLearning
- 7,257
- 4
- 33
- 68
1
vote
0 answers
FastAPI Huggingface Inference every request increases CPU RAM usage
We have a FastAPI server, and try to use a pretrained HuggingFace model for inference (on a vast.ai gpu server). The Inference accepts one or many images, and returns an array with the image links provided and the processed output. Neither FastAPI…
Guntram
- 414
- 2
- 5
- 18
1
vote
1 answer
Add a Classification Head on Top of Huggingface Vilt Model
I want to add a classification layer in pytorch on top of the huggingface vilt transformer, so that I can classify my text labels.
Generally in normal settings vilt takes an image, question pair and outputs the answer of the question after forward…
user10418143
- 220
- 3
- 11
1
vote
0 answers
Deepspeed tensor parallel gets problem in tensor alignment when using tokenizer
I tried to use deepspeed to conduct tensor parallel on starcoder as I had multiple small GPUs and each of which cannot singly hold the whole model.
from transformers import AutoModelForCausalLM, AutoTokenizer
import os
import torch
import…
ddaa
- 49
- 2
1
vote
0 answers
Invalid key: 409862 is out of bounds for size 0
How I can fix this:
I writed code for training GPT-2 on dataset by Hugging Face, but I have an error and don't know why I got this error:
---------------------------------------------------------------------------
IndexError …
Vovancho
- 11
- 2
1
vote
0 answers
How to use textual entailment model generated by fine-tuning STS models on HuggingFace
I am trying to fine-tune an STS model for Textual Entailment classification for "entailment", "neutral", and "contradiction".
Here is the source code available on HuggingFace Sentence-BERT NLI:…
miner2008
- 11
- 2
1
vote
0 answers
Discrepancies: Sagemaker vs. Local Hugging Face Model inference Results
I am using a Hugging Face model (sentence-transformers/distiluse-base-multilingual-cased-v2) on my local system to convert text into vectors.
class SequenceEncoder(object):
def __init__(self, device=None):
self.device = device
…
Oded
- 336
- 1
- 3
- 17
1
vote
2 answers
does hugging face model.generate for flan-T5 default is summarization?
Given the following code. why does the function:
model.generate()
returns a summary, where does it order to do summary and not some other task? where can I see the documentation for that as well.
model_name = ‘google/flan-t5-base’
model =…
user552231
- 1,095
- 3
- 21
- 40
1
vote
0 answers
AttributeError: module 'torch.nn.init' has no attribute 'trunc_normal_' While saving trasnformer google/vit-base-patch16-224-in21k model to local
I am trying to save google/vit-base-patch16-224-in21k locally and then upload it to s3 for hosting this model in the SageMaker environment.
from transformers import AutoImageProcessor, ViTModel
# tokenizer =…
iamabhaykmr
- 1,803
- 3
- 24
- 49
1
vote
2 answers
How does one use accelerate with the hugging face (HF) trainer?
What are the code changes one has to do to run accelerate with a trianer?
I keep seeing:
from accelerate import Accelerator
accelerator = Accelerator()
model, optimizer, training_dataloader, scheduler = accelerator.prepare(
model, optimizer,…
Charlie Parker
- 5,884
- 57
- 198
- 323
1
vote
0 answers
How to substitute the OpenAiEmbeddings with Huggingface on Langchain?
const { HuggingFaceInferenceEmbeddings } = require('@huggingface/inference');
const embeddings = new HuggingFaceInferenceEmbeddings({
apiKey: process.env.HUGGINGFACEHUB_API_KEY,
model: "hkunlp/instructor-large",
});
vectorStore = await…
user42141
- 33
- 4
1
vote
0 answers
Why do you need to re-upcast the norm layers of HF falcon to 32 floating point (fb32) when the code use floating point 16 (fb16)?
I saw these lines:
bnb_4bit_compute_dtype=torch.float16,
...
optim = "paged_adamw_32bit"
...
for name, module in trainer.model.named_modules():
if "norm" in name:
module = module.to(torch.float32)
in the falcon tutorial. These are…
Charlie Parker
- 5,884
- 57
- 198
- 323