0

I'm working on moving all documents in SOLR to elasticsearch using Python code with Pysolr.

With pysolr i'm able to access only 499 documents, it there a way to access the entire set of documents.

pjesudhas
  • 399
  • 4
  • 13

1 Answers1

0

The documents could be accessed in sets of 499 using the offset and size parameter in pysolr API, i found this after checking the Pysolr api more in detail. The sample code is attached below :

limit = 499
offset = 0
while True:
    kwargs = {'sort':'id DESC'}
    response = source.search(q='projectid:774 AND source:FACEBOOK', start=offset, rows=limit, **kwargs  )
    print(len(response.docs))

    offset = offset + limit

    if len(response.docs) == 0:
        break

    # Do neccessary Operations
    pass
pjesudhas
  • 399
  • 4
  • 13