How to remove input from from generated text in GPTNeo?

Question

I'm writing a program to generate text... I need to remove the input from the generated text. How can I do this? The code:

input_ids = tokenizer(context, return_tensors="pt").input_ids
gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.8,
    top_p=0.9)
strs = tokenizer.batch_decode(gen_tokens)[0]

Here the strs contains the input I've given... How to remove that?

score 0 · Answer 1 · answered Mar 24 '22 at 13:04

0

The Transformers library does not provide you with a way to do it, but this is something you can easily achieve with 1 line of code:

strs = strs.replace(context,"")

This is actually what I'm doing behind my NLP Cloud API as it uses Transformers behind the hood.

answered Mar 24 '22 at 13:04

Julien Salinas

1,059
1
10
23

1

It doesn't work always. It fails, I don't know why the output is changed. I used strs = strs[len(context):len(strs)] It wasn't working for sometime. – Crawl Cats Admin Apr 09 '22 at 02:40
You are founder? oww... Thats nice! – Crawl Cats Admin Apr 09 '22 at 02:42
1

I am ;) I sometimes noticed that, on rare occasions, GPT-Neo/GPT-J changes the input during text generation. It happens in case of wrong punctuation. For example if there's "hello , friend" in your input, it will be turned into "hello, friend" by the model. Maybe that's what is happening to you too? – Julien Salinas Apr 11 '22 at 14:13
I did also see! – Crawl Cats Admin Jun 04 '22 at 11:14

How to remove input from from generated text in GPTNeo?

1 Answers1