llamaindex is a method of randomly loading a lot of related information, or a method of shaping information so that it is easy for llamaindex to recognize, which is preferable?
Currently I'm using llamaindex with 1,000 text files (5,000,000 characters) loaded. However, all of these text files read the text of articles on a certain homepage as they are, and they are not shaped so that llamaindex can easily recognize them.
Currently, even if you ask llamaindex a question related to an article that has already been loaded, it may return that there is no irrelevant answer or information, so we would like to improve it.
Therefore, we will tell you which is the better method, whether to read 100 times the number of articles related to the articles that have already been read to cover the amount, or to arrange the existing 1,000 articles manually and improve the quality. Can I have it?
llamaindex is used by calling it as a tool from langchain.
I would appreciate it if you could point out the cost (time and expense), accuracy, execution time, etc. If there is any other better way, I would appreciate it if you could tell me even just the related words. Please let me know if there are any points that are difficult to understand.
I've had llamaindex load 1,000 text files, but sometimes asking related questions doesn't return accurate information.
