0

I recently started to consider CouchDB to store a large list of dicts I manipulate in Python. Large in my case means about 20k elements.

I found out that the update() method is very fast in pushing my whole list in one HTTP call. It takes about 3 seconds, which is perfect for my case.

I also need (in a different script) to retrieve the whole content of this database and store it in a list of dicts. This is the opposite operation to the one above. Unfortunately the only way I found to do this is via

# db is the database opened following a couchdb.Server() call
mylist = list()
for id in db:
    mylist.append(db[id])

This takes 10 minutes as there is a call for each element of the loop.

  • Is there an equivalent to update() to bulk-retrieve the contents of a database?
  • or should I approach the retrieving part in a different, more efficient way?
WoJ
  • 27,165
  • 48
  • 180
  • 345
  • The provided answer is perfect, I found in the meantime a duplicate and will flag my question (self-whipping style) – WoJ Oct 18 '14 at 11:00
  • possible duplicate of [Multiple CouchDB Document fetch with couchdb-python](http://stackoverflow.com/questions/1640054/multiple-couchdb-document-fetch-with-couchdb-python) – WoJ Oct 18 '14 at 11:00

1 Answers1

0

The easiest way to get every document is to issue a request to _all_docs and pass include_docs=true.

_all_docs will get the basic details (_id, revision) of all documents in the database. include_docs causes the full documents to be included in the response.

/mydatabase/_all_docs?include_docs=true

If you need something more subtle than returning entire documents (e.g. returning a single scalar value for each document) then you need to look into views.

Ant P
  • 24,820
  • 5
  • 68
  • 105
  • speaking in context of couchdb-python, you need to call `db.view("_all_docs", include_docs=True)` or better `db.iter_view("all_docs", 100, include_docs=True)` to not fail with OOM error since this call will return you mostly whole database data. – Kxepal Oct 18 '14 at 06:43