I'm new to huggingface and am working on a movie generation script. So far my code looks like this
from transformers import GPT2Tokenizer, GPTNeoModel
from datasets import load_dataset
dataset = load_dataset('text',data_files={'train':['youtube_3/script.txt']})
tokenizer = GPT2Tokenizer.from_pretrained('EleutherAI/gpt-neo-1.3B')
model = GPTNeoModel.from_pretrained('EleutherAI/gpt-neo-1.3B')
However I keep getting this error
ValueError: Please pass `features` or at least one example when writing data
Does this have anything to do with the way I define my tokenizers and such.
How would I fix this? Any help would be appreciated.