Stack Overflow
Stack Overflow

Questions tagged [faiss]

77 questions
0
votes
0 answers

Issue retrieving specific data from documents embedded into a vectorstore in langchain

I'm looking for help about retrieving data from documents embedded in a vectorstore. I'm still pretty new to this, and I may miss something obvious. The issue I'm facing is that some specific data from the documents don't seem to be found when using…
JDupont1984
  • 1
  • 2
0
votes
0 answers

CMake variables error when installing faiss-gpu: "Failed to detect a default CUDA architecture"

I’m order to install faiss-gpu, I first needed to install cmake and did it by cloning the cmake repo from github.com/Kitware/CMake and running ./bootstrap && make && make install Then to install faiss-gpu I cloned the following repo…
Jose Cruz
  • 1
0
votes
0 answers

langchain vectorstore question and answer from a single embedding in vectorstore

I have worked in creating a vectorstore from a series of paragraphs from a text document. The text of the document has been splitted in non-overlapping paragraphs for a good reason, as these represents different informations. These paragraphs have…
Fernando Delgado Chaves
  • 51
  • 6
0
votes
1 answer

FAISS vectorstore created with LangChain yields AttributeError: 'OpenAIEmbeddings' object has no attribute 'deployment' / 'headers'

For the past few weeks I have been working at a QA retrieval chatbot project with LangChain and OpenAI in Python. I have an ingest pipepline set up in a notebook on Google Colab, with which I have been extracting text from PDFs, creating embeddings…
Bert
  • 1
  • 1
0
votes
0 answers

Segmentation fault while using FAISS `write_index`

I am trying to create an index by using FAISS to the specified path on Docker container. This happens within Python script which is executed during container creation. The code for this is shown below (some of the functions' definitions were removed…
Daniyar Absatov
  • 1
  • 1
0
votes
0 answers

Can managed vector db like pinecone support MMR

To get diversity in the search results, I want to use max marginal relevance for this. I want to fetch N documents form vector db similar to question, then subsequently I want to get documents which are similar to question but dissimilar to existing…
raju
  • 4,788
  • 15
  • 64
  • 119
0
votes
0 answers

How to get nearest vector in every cluster of faiss.IndexIVFFlat

I'm using faiss.IndexIVFFlat with nlist=10 to make an index of vectors I want to search. I have a query vector xq and I want to get the vector nearest to xq in each of the 10 clusters in the index. Is there a way to do this with faiss? I see here it…
whitewaterbottle
  • 1
0
votes
1 answer

Why a Faiss index works on macOS but not on Docker (TypeError in method)?

I'm trying to search for vectors using Faiss and access them through an API. All my packages are installed with Conda. def get_similar_posts(embeddings: list[float]) -> list[str]: toSearch = np.array([embeddings]) toSearch.reshape(1,…
Julien
  • 1
  • 1
0
votes
1 answer

How to combine multiple FAISS indexes into one to get a single retriever

pdf = load_pdf(help_doc_name) faiss_index_ft9Help = FAISS.from_documents(pdf, OpenAIEmbeddings()) faiss_index_ft9Help.save_local(index_path + "/" + help_doc_name) # load newsletters pdf = load_pdf(newsletters_doc_name) faiss_index_newsletters =…
Pasindu Lakshan
  • 80
  • 5
0
votes
1 answer

Why am I getting an authentication error when trying to run a LangChain tutorial on FAISS vector database with OpenAI API?

I follow a YouTube LangChain tutorial where it teaches Create Your Own ChatGPT with PDF Data in 5 Minutes (LangChain Tutorial) and here is the colab notebook link provided by the author for his work below the video description. I didn't modify a lot…
林抿均
  • 33
  • 7
0
votes
0 answers

Error importing FAISS on Lambda when a Numpy layer is added

I have FAISS running on a lambda layer, as soon as I add a layer containing Numpy it gives me this error, anyone know how to tackle this? Test Event Name test Response { "errorMessage": "Unable to import module 'lambda_function': cannot import…
PPython
  • 1
  • 1
0
votes
1 answer

FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

I trained a HSMW Index using faiss on Windows. I wanted to deploy this to an Azure Function App, hence I just pip to install faiss-cpu==1.7.3 rather than the recommended install with conda. The code that searches for the most similar vector…
Edgar H
  • 1,376
  • 2
  • 17
  • 31
0
votes
0 answers

HNSW index on BigQuery

Is there a way to create and incrementally update a BigQuery ANN index like HNSW? I'm interested in using BigQuery for vector storage, but trying to avoid full-scans. I know BigQuery originally was "full scan, all the time," but they introduced…
jamesvillarrubia
  • 51
  • 6
0
votes
0 answers

Adding 4M embeddings to Faiss Index

I'm learning Faiss and trying to build an IndexFlatIP quantizer for an IndexIVFFlat index with 4000000 arrays with d = 256. My code is as follows: import numpy as np import faiss d = 256 # Dimension of each feature vector n = 4000000 # Number of…
Johnny
  • 117
  • 10
0
votes
0 answers

How do I resolve missing ARROW_PYTHON_INSTALL_DIR on MacOS?

My end goal: Install autofaiss such that it can be imported as a library in python3.11 on MacOS X Ventura. I think the main problem I'm running into is that I don't know how to debug pip3 / brew / cmake errors on MacOS -- where are the respective…
Jon Watte
  • 6,579
  • 4
  • 53
  • 63
1 2 3
4
5 6