How to do casbah bulk inserts

Question

In casbah I see the following function defined on a MongoCollection:

def
insert[A](docs: A*)(implicit dbObjView: (A) ⇒ commons.TypeImports.DBObject, concern: mongodb.WriteConcern = writeConcern, encoder: TypeImports.DBEncoder = ...): TypeImports.WriteResult

If I understand this right you can pass a number of DBObjects to this function for a bulk insert.

My concern/question is that this is a list of parameters. What if I have a huge bulk insert (say a couple hundred)? Is there a limit of object that can be passed this way?

I don't understand the question fully. Do you ask how many arguments can be passed to the first parameter list (or to varargs in general)? — kiritsuku, May 15 '14 at 19:28

score 3 · Accepted Answer · answered May 16 '14 at 13:06

There is a limit on the size of the document that can be used in a single command ~16mb so there is a limit on how many inserts you can do via that route.

However, if you use one of the BulkOperationBuilders released in Casbah 2.7 for MongoDB 2.6 it will automatically split the operation into batches so you can do larger bulk operations:

val collection = MongoClient()("test")("bulkOperation")
collection.drop()

// Ordered bulk operation
val builder = collection.initializeOrderedBulkOperation
builder.insert(MongoDBObject("_id" -> 1))
builder.insert(MongoDBObject("_id" -> 2))
builder.insert(MongoDBObject("_id" -> 3))

builder.find(MongoDBObject("_id" -> 1)).updateOne($set("x" -> 2))
builder.find(MongoDBObject("_id" -> 2)).removeOne()
builder.find(MongoDBObject("_id" -> 3)).replaceOne(MongoDBObject("_id" -> 3, "x" -> 4))

val result = builder.execute()

There are ordered or unordered operations see: http://mongodb.github.io/casbah/whats_new.html#ordered-unordered-bulk-operations

How to do casbah bulk inserts

1 Answers1