I am new to the world of LangChain and LLama Index. I am working on a project for document consultation (building regulations) using LLMs. I noticed that when, for example, on LLama Index, I use the query engine, the responses are based solely on the ingested data. However, when I use the chat engine, the LLM also draws (if not solely) from its background knowledge. I would like to build a chatbot that only draws from my documents.
Does anyone know why? And how to resolve this issue?
Here is the code:
documents = SimpleDirectoryReader('./data').load_data()
parser = SimpleNodeParser()
nodes = parser.get_nodes_from_documents(documents)
index = GPTVectorStoreIndex(nodes, show_progress = True)
query_engine = index.as_query_engine()
query = input()
response = query_engine.query(query)
print(response)
In this case if i asks something like "Who is Cristoforo Colombo?" the response is correctly "i don't know".
On the other hand, with the following code:
documents = SimpleDirectoryReader('./data').load_data()
parser = SimpleNodeParser()
nodes = parser.get_nodes_from_documents(documents)
index = GPTVectorStoreIndex(nodes, show_progress = True)
service_context = ServiceContext.from_defaults(llm=OpenAI(model="gpt-3.5-turbo-0613"))
chat_engine = index.as_chat_engine(chat_mode="openai", verbose=True, service_context = service_context)
index.as_chat_engine(chat_mode="openai", verbose=True, service_context = service_context)
query =''
while query != "stop":
query = input()
response = chat_engine.chat(query)
print(response)
The answer seems to be taken from its background knowledge, and it knows who Christopher Columbus is
