can we have memory preservation for chatting in llama-index?

Question

I would like to know whether llama-index can be used for creating a custom chatgpt based on the documentation or not. I have an ample amount of data, however, I want to retain context, and add answer the questions accordingly. Is it possible to do that ? if yes then how ? any leads would be really helpful

Answer 1

Yes llama_index can be used to create custom chatbot. There are many online articles and videos explaining this process.

First need to import the below packages.

from llama_index import ServiceContext, SimpleDirectoryReader, GPTVectorStoreIndex, LLMPredictor, PromptHelper, StorageContext, load_index_from_storage
from langchain.chat_models import ChatOpenAI
import os

We need to set the open ai key as below:

os.environ["OPENAI_API_KEY"] = "your api key"

We need to start by creating a vector store based on the data available. Can use the below code to create a vector store. In the below the local data files are stored as pdfs in the folder "Knowledge" and the created index will be stored in the folder "storage".

def construct_index():
    def construct_index():
    max_input_size = 4096
    num_outputs = 512
    max_chunk_overlap = 1
    chunk_size_limit = 1000

    print("Index building started")
    prompt_helper = PromptHelper(
        max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

    llm_predictor = LLMPredictor(llm=ChatOpenAI(
        temperature=0.1, model_name="gpt-3.5-turbo", max_tokens=num_outputs))

    service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor,
        prompt_helper=prompt_helper)

    # load data from the folder
    documents = SimpleDirectoryReader("Knowledge").load_data() 

    index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

    index.storage_context.persist("storage")

    print("Index created successfully!")

Once the index is created, questions related to the local data can be asked by querying this index. indexPath variable refers to the folder where the vector index is stored.

def post():
    args = parser.parse_args()
    indexPath = 'storage'

    storage_context = StorageContext.from_defaults(persist_dir=indexPath)
    index = load_index_from_storage(storage_context)
    query_engine = index.as_query_engine()
    prompt = args['q']
    response = query_engine.query(prompt)

    return response.response

It is important to note that the quality and accuracy of answers depends on the quality of the content fed.