Questions tagged [fine-tune]
156 questions
2
votes
1 answer
how to train a bert model from scratch with huggingface?
i find a answer of training model from scratch in this question:
How to train BERT from scratch on a new domain for both MLM and NSP?
one answer use Trainer and TrainingArguments like this:
from transformers import Trainer,…
Jack.Sparrow
- 121
- 1
- 3
- 8
2
votes
1 answer
Semantic Search fine-tune
eg. Pre-Trained BERT Result for sentence cosine similarity
======================
Query: milk with chocolate flavor
Top 10 most similar sentences in corpus:
Milka milk chocolate 100 g (Score: 0.8672)
Alpro, Chocolate soy drink 1 ltr (Score:…
Juned Ansari
- 5,035
- 7
- 56
- 89
2
votes
0 answers
how to fine-tune "distiluse-base-multilingual-cased" model for text similarity customisation
I am trying to do semantic search but pre-trained model is not accurate on Italian grocery data.
eg.
Query: latte al cioccolato #chocolate milk
Top 3 most similar sentences in the corpus:
Milka cioccolato al latte 100 g (Score: 0.7714) #Milka…
Juned Ansari
- 5,035
- 7
- 56
- 89
1
vote
0 answers
Fine tuning Sentence transformers for semantic product search task
Problem I have at hand is to build a product suggestion model which suggest products based on the context of the search query of a user. My plan is to get a pre-trained model from the sentence-transformers pre-trained models and embed product…
L.D. WEERARATHNE
- 11
- 1
1
vote
1 answer
Fine tune an LLM NOT on question/answer dataset
Most of the material out there for tine tuning LLMs use a question/answer dataset for fine tuning. Problem is, that's not my use case. I would like to fine tune an LLM on domain knowledge which exists as a set of documents and that set can't really…
Demiurg
- 1,597
- 8
- 26
- 40
1
vote
0 answers
GPT2 LLM fine-tuned model not generating expected answer
I am finetuning gpt2 model to answer questions with given faq.json.
There is some issue with the answer generated by below code. I am assuming I have not done encoding/decoding of questions and answers correctly.
Code -
import torch
from…
tagg
- 383
- 4
- 7
1
vote
1 answer
OpenAI Fine-Tuning error - 'fileName contains an invalid filename: wrong suffix.'
I am trying to fine-tune a GPT model through the Azure OpenAI API.
I now need to upload the file to openai using the code below:
file_name = "training_data_prepared.jsonl"
upload_response = openai.File.create(
file=open(file_name, "rb"),
…
Curtis
- 21
- 3
1
vote
1 answer
How do I fine tune BERT's self attention mechanism?
My goal is to fine tune BERT's self attention so that I can see to what extent two random sentences in a document (with positional encoding) rely on each other contextually.
Many explanations and article that I see talk about the implementation of…
AjS
- 13
- 4
1
vote
0 answers
OpenAI fine-tuning training data exceeds the token limit
I am using curie model to fine-tune in Python. Basically, I am passing the training data of form {"prompt":"completion"} and I have 736 prompt-example pairs.
My example completions are pretty long - I aim at generating a JSON file based on a…
user1180944
- 11
- 2
1
vote
0 answers
mT5 Question/Answering fine tuning is generating empty sentences during inference
mT5-small Question Answering training is converging to high accuracy, high validation accuracy, near-zero low loss; however, when testing the model on trained questions, I am always receiving empty answers.
Experiment Language: Arabic
Dataset used:…
Moustafa Banbouk
- 73
- 1
- 5
1
vote
0 answers
Fine-tune SentenceTransformer/SBERT for Extractive Text Summarization
Newbie here on NLP.
I want to build extractive text summarization, try to read this https://huggingface.co/blog/how-to-train-sentence-transformers, I think there is a way to fine-tune the model with my own dataset (data, and language), in case 2…
Python Beginner
- 53
- 7
1
vote
1 answer
GPU out of memory fine tune flan-ul2
OutOfMemoryError: CUDA out of memory. Tried to allocate 256.00 MiB
(GPU 0; 15.78 GiB total capacity; 14.99 GiB already allocated; 3.50
MiB free; 14.99 GiB reserved in total by PyTorch) If reserved memory
is >> allocated memory try setting…
Salty Gold Fish
- 431
- 5
- 14
1
vote
2 answers
Expected file to have JSONL format, where every line is a JSON dictionary. openai createFile for fine tune
I created file with name mydata.jsonl and I put on it these lines
{
"prompt": "aa",
"completion": "bb"
}
{
"prompt": "cc",
"completion": "dd"
}
then in index.js I did this function
const {…
Fatma Mahmoud
- 89
- 2
- 13
1
vote
1 answer
Why does LLM(LLaMA) loss drop staircase-like over epochs?
I'm training a LLM(LLaMA-6B) and have noticed that its loss seems to drop in a stair-like fashion over the epochs. Specifically, I'll see little loss change for one epoch, and then suddenly the loss will drop quite a bit after a new epoch.
I'm…
Jing zhao
- 11
- 1
1
vote
1 answer
Fine tuning flair ner model
I am trying to fine tune flair ner model using these lines of code:
embedding_types = [WordEmbeddings('glove'),
WordEmbeddings('extvec'),
WordEmbeddings('crawl'),]
embeddings =…
nasr el hamzaoui
- 11
- 1