I have been trying to fine-tune the T5 model on Conversational Machine Comprehension Datasets with no satisfactory results. Particularly, I have been trying it on QuAC, with unanswerable questions. I am a beginner in NLP tasks and do not know how to go about it. I have extracted the history, context, questions and answers. Should the answers be prepended or appended to the context? Another problem that I encountered was that of max_input_seq. The input text is truncated, which I believe is the reason I am not getting the expected results.
I have tried prepending the history with tags separating the history and context, with the question coming first. The question, history and context are all combined into the input text. I tokenized the inputs without providing the maximum input sequence using HuggingFace's T5 tokenizer and T5ForConditionalGeneration, but I am not able to obtain an F1 score of above 0.25.
