3

Whole day I have worked with available text generation models

Here you can find list of them : https://huggingface.co/models?pipeline_tag=text-generation&sort=downloads

I want to generate longer text outputs, however, with multiple different models, all I get is repetition.

What am I missing or doing incorrectly?

I will list several of them

Freshly released meta GALACTICA - https://huggingface.co/facebook/galactica-1.3b

The code example

from transformers import AutoTokenizer, OPTForCausalLM

tokenizer = AutoTokenizer.from_pretrained("facebook/galactica-1.3b")
model = OPTForCausalLM.from_pretrained("facebook/galactica-1.3b", device_map="auto")

 
input_text = "The benefits of deadlifting\n\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids,new_doc=False,top_p=0.7, max_length=1000)
print(tokenizer.decode(outputs[0]))

The generated output

enter image description here

Facebook opt - https://huggingface.co/facebook/opt-350m

The tested code

from transformers import GPT2Tokenizer, OPTForCausalLM

model = OPTForCausalLM.from_pretrained("facebook/opt-350m")
tokenizer = GPT2Tokenizer.from_pretrained("facebook/opt-350m")

prompt = "The benefits of deadlifting can be listed as below:"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
generate_ids = model.generate(inputs.input_ids, max_length=800)
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

The generated output

enter image description here

Rubén
  • 34,714
  • 9
  • 70
  • 166
Furkan Gözükara
  • 22,964
  • 77
  • 205
  • 342

1 Answers1

-1

The default strategy for generating text seems to quickly result in repetitions. See e.g. here for an analysis of the effect of various parameters to obtain less repetitive results:

https://www.kaggle.com/code/tuckerarrants/text-generation-with-huggingface-gpt2

user21291703
  • 1
  • 1