I have approximately 250,000 JSON-formatted files, each with one object in it (formatted just how CouchDB likes it with _id). What's the best way to import these into my remote CouchDB server as records?
-I am on a windows xp machine.
-I have internet access but I can't set up a couchDB server on my local machine and have it be WWW accessible (firewall constraints.) so no easy replication.
I would highly suggest that you look into the bulk doc API in the couchdb wiki: http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
Basically, you make a POST request to /someDatabase/_bulk_docs that looks like this:
{
"docs": [
{ "_id": "awsdflasdfsadf", "foo": "bar" },
{ "_id": "cczsasdfwuhfas", "bwah": "there" },
...
]
}
Just like any other POST request, if you don't include _id properties, couchdb will generate them for you.
You can use this same operation to update a bunch of docs: just include their _rev property. And if you want to delete any of the docs that you are updating, then add a "_deleted": true property to the document.
If you have a json file with your documents and use curl, it could look like:
curl -H "Content-Type: application/json" --data-binary @/home/xxx/data.json https://usr:pwd@host:5984/someDatabase/_bulk_docs/
Cheers.
