Stack Overflow
Stack Overflow

Questions tagged [llm]

A general tag for large language model (LLM)-related subjects. Please ALWAYS use the more specific tags if available (GPT variants, PaLM , LLaMa, BLOOM, Claude etc..)

A general tag for large language model (LLM)-related subjects. Please ALWAYS use the more specific tags if available (GPT variants, PaLM , LLaMa, BLOOM, Claude etc..)

A large language model is characterized by its large size. Their AI accellerator networks are able to process huge amounts of text data, usually scraped from the internet.

200 questions
0
votes
1 answer

ChromaDB limit queries by metadata

I have a ChromaDB that has "source_type" = 'guideline' | 'practice' | 'open_letter'. If my k/p_value is the default of 6, is there a way I can limit my similarity search first based on "source_type", THEN get the 6 pieces of evidence? Is this a…
Mark
  • 241
  • 3
  • 14
0
votes
0 answers

ChromaDB Management

Is there an efficient way of managing ChromaDB instances in place - specifically using metadata to delete documents? Inserts/upserts are supported. Deletes by collection are supported, but I can't find much in their API for deleting specific…
Mark
  • 241
  • 3
  • 14
0
votes
0 answers

How to use results in output directory in producing environment?

I have trained facebook/opt-6.7b with [BELLE][1]'s dataset, however the biggest file optimizer.pt in qlora/output/checkpoint is only 1.1GB, and size of the whole facebook/opt-6.7b is about 13GB. so where is the real model and how to use it in…
forestbat
  • 453
  • 4
  • 10
0
votes
0 answers

tf.compat.v1.estimator.Estimator(): NameError: name 'model_fn' is not defined

I am trying to create a pet LLM using GPT-2 following instructions here: https://thomascherickal.medium.com/how-to-create-your-own-llm-model-2598615a039a The code gives syntax error while calling tf.compat.v1.estimator.Estimator() with model_fn as…
sm535
  • 587
  • 7
  • 20
0
votes
0 answers

How to measure accuracy in Langchain + OpenAI project

We created something similar to internal chatbot, where we processed a lot of internal documents. We have testing dataset with question and correct answer. Right now we are focusing on better accurancy and we are trying to find a way how to compare…
Radek Vicha
  • 1
0
votes
0 answers

LMM Fine Tuning - Supervised Fine Tuning Trainer (SFTTrainer) vs transformers Trainer

When should one opt for the Supervised Fine Tuning Trainer (SFTTrainer) instead of the regular Transformers Trainer when it comes to instruction fine-tuning for Language Models (LLMs)? From what I gather, the regular Transformers Trainer typically…
Marvin Martin
  • 1
0
votes
0 answers

How to size compute/gpu/storage/network for generative AI or LLM?

I would like to provision compute (servers), gpus (say 2 A100 80GB or H100), storage and network (may be 100GbE) to run OpenApaca 7B (https://huggingface.co/openlm-research/open_llama_7b) model. How do I go about sizing this? AWS/GCP cluster sizing…
techele
  • 165
  • 3
  • 8
0
votes
1 answer

How to use a Llama model with langchain? It gives an error: Pipeline cannot infer suitable model classes from: - HuggingFace

finetuned a model (https://huggingface.co/decapoda-research/llama-7b-hf) using peft and lora and saved as https://huggingface.co/lucas0/empath-llama-7b. Now im getting Pipeline cannot infer suitable model classes from when trying to use it along…
Lucas Azevedo
  • 1,867
  • 22
  • 39
0
votes
1 answer

use llama index to create embeddings for commercial pipeline

I have the the python 3 code below. In the code I am using llama_index from meta to create an index object from my own text corpus. I'm then passing queries to that index object to get responses back from openai's chatgpt, using my additional text…
user3476463
  • 3,967
  • 22
  • 57
  • 117
0
votes
1 answer

How to handle token limit in ChatGPT3.5 Turbo when creating tables?

End user can copy tables from a pdf like , paste the text in openai playground bird_id bird_posts bird_likes 012 2 5 013 0 4 056 57 70 612 0 12 and will prompt the gpt with "Create table with the given text" and gpt generates a table like…
usr_lal123
  • 650
  • 12
  • 28
0
votes
0 answers

I try to use GPTJ-lora model to generate txt, but the max-length of the generated text seemed to be 20 tokens. How to make it longer

import transformers #from transformers import AutoModelWithHeads model.load_adapter("./",adapter_name='lora') peft_model_path="./" tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B") tokenizer.pad_token =…
Dapeng Zhang
  • 1
  • 3
0
votes
1 answer

Streaming response line chatgpt

Does anyone know if I can display chatgpt-like streaming response in Streamlit using streamlit_chat message? I need something like message(streaming=True) or any other alternative for this. my code segment is as below: from streamlit_chat import…
iamamir
  • 1
0
votes
1 answer

Output 0 of DequantizeAndLinearBackward is a view and is being modified inplace. This view was created inside a custom Function and the autogrid

I am trying to fine-tune GPT J, but I have this error. I think it's related to the activation function and it's in-place but I don't know how to code it to fix it. Is it a parameter inside the activation function that needs to be disabled? If yes,…
May Ouir
  • 21
  • 2
0
votes
0 answers

SimpleTransformers Can't Find GPT_PERSONA_CHAT

So I'm using simple transformers and instantiating a pretty basic ConvAi model. from simpletransformers.conv_ai import ConvAIModel train_args = { "num_train_epochs": 50, "save_model_every_epoch": False, } model =…
DevDog
  • 111
  • 2
  • 9
0
votes
0 answers

Aitextgen doesn't generate any new text

So I'm using the aitextgen library to finetune gptneo. However once I did that and generated some text, it just spewed out the exact same sentences as in the text. from aitextgen import aitextgen ai = aitextgen(model="EleutherAI/gpt-neo-125M",…
DevDog
  • 111
  • 2
  • 9
1 2 3
…
13 14