I'm having an issue using Llama_Index to use an index previously generated for custom content ChatGPT queries.
I generated the index with the following code:
from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTSimpleVectorIndex, LLMPredictor, PromptHelper, ServiceContext
from langchain import OpenAI
import sys
import os
def construct_index():
# set maximum input size
max_input_size = 40960
# set number of output tokens
num_outputs = 20000
# set maximum chunk overlap
max_chunk_overlap = 20
# set chunk size limit
chunk_size_limit = 600
# define prompt helper
prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
# define LLM
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-davinci-003", max_tokens=num_outputs))
documents = SimpleDirectoryReader("./data").load_data()
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
index.save_to_disk('./index.json')
return index
To use query something the code I use is the following:
def ask_ai(query):
index = GPTSimpleVectorIndex.load_from_disk('./index.json')
response = index.query(query)
return response.response
This is a popular code that in fact works when I run it in a virtual environment (py ./index.py) adding a line to call by default the construct_index or ask_ai methods.
However, when I try to put it behind an HTTP request, like in an AWS Lambda or Flask API, the ask_ai fails in the load_from_disk method for both approaches with the same error:
ERROR:app:Exception on /api/query [POST]
Traceback (most recent call last):
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "C:\pmi\python-flask-restapi\app.py", line 17, in qyery_chatgpt
return appService.ask_ai(query)
File "C:\pmi\python-flask-restapi\app_service.py", line 10, in ask_ai
index = GPTSimpleVectorIndex.load_from_disk('index.json')
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\base.py", line 352, in load_from_disk
return cls.load_from_string(file_contents, **kwargs)
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\base.py", line 328, in load_from_string
return cls.load_from_dict(result_dict, **kwargs)
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\vector_store\base.py", line 260, in load_from_dict
return super().load_from_dict(result_dict, **config_dict, **kwargs)
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\base.py", line 304, in load_from_dict
docstore = DocumentStore.load_from_dict(result_dict[DOCSTORE_KEY])
File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\docstore.py", line 72, in load_from_dict
for doc_id, doc_dict in docs_dict["docs"].items():
KeyError: 'docs'
For the implementation of Flask I took it from here:
https://github.com/bbachi/python-flask-restapi
Just by leaving a single method in the AppService (the ask_ai) and invoking it from a single route in the controller. The docker file for the AWS container-based lambda is:
FROM public.ecr.aws/lambda/python:3.10
COPY requirements.txt ./
RUN pip3 install -r requirements.txt
COPY myfunction.py ./
COPY index.json ./
CMD ["myfunction.lambda_handler"]
In both cases the index.json holds the index generated with the above-provided code and for the lambda the myfunction.py contains the ask_ai method implementation behind the lambda_handler method.
Has someone faced such issue? any help is much appreciated in advance!
