1

I'd like to use Llama to do a conversational chat bot to answer questions on scientific news articles. Can I use a collection of unlabelled scientific articles to fine tune Llama and to "increase its vocabulary"/"extend its knowledge base"?

I'm not sure how to approach this task without manually creating extra texts as labels for designed elaboration/summarization prompts created by me.

Ong Hai Xiang
  • 11
  • 2
  • Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. – Community Jun 07 '23 at 18:45
  • Fine-tuning is, by definition, a supervised process, so I guess what you are suggesting is not feasible. You could use unlabelled data to perform a further pre-train, possibly. – SilentCloud Jun 30 '23 at 14:16

2 Answers2

0

In my experience, you cannot. We recently worked on llms at grandeur. We worked with both Palm and GPT3. Fine tuning is about training the model with some context. Like you you are required to give the model labeled examples.

We tested fine tuning a lot, but we realized that fine tuning is not quite effective until you provide a lot of data. Like 100s of examples, if not thousands ref.

Instead, I recommend using the vector db based method to enhance the response of models. You can read more it from this amazing article https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/

Moiz Husnain
  • 50
  • 11
0

I'm looking to do a similar thing and it should be possible but it's not cheap or easy. Check out this paper: https://arxiv.org/abs/2304.08177

These guys did what you intend to do. First they "retrained" the Llama with a corpus of around 20GB of Chinese text, and then they fine-tuned the model on the Chinese translation of the Alpaca dataset. In the first retraining they even increased the vocabulary size of the model.

I'm saying "retrained" because they used LoRA for both the pre-training and for the finetuning, so it's not exactly pre-training as the original model weights are not updated. However, they added enough new weights using LoRA that they achieved an improvement in the understanding of the Chinese language.

Still, I agree with Moiz Husnain in that in your case, this is probably both a harder and inferior approach. Especially if the language in which your chatbot will converse is English.

Levara
  • 1,058
  • 2
  • 12
  • 21