Shard key and Azure CosmosDB for MongoDB API

Question

I created a CosmosDB database that uses MongoDB's Python SDK, let's call the database as school and the collection as students. While creating the database, the Azure UI asked me to create a shard key for unlimited storage capacity, let's call it school.students.

I am using pymongo to add the following document:

{
  "id": "abc123",
  "name": "jack"
}

and the code as:

from pymongo import MongoClient

student_data = {
  "id": "abc123",
  "name": "jack"
}

client = MongoClient("mongodb://...")
db = client["school"]
collection = db["students"]
collection.insert_one(student_data)

When I chose the storage with 10GB, I have no problem adding the document, but when I chose the unlimited option with shard key, I get the following error:

pymongo.errors.WriteError: document does not contain shard key at 'school.students'

Question is, where should I use the shard key? and how to get over this?

score 5 · Accepted Answer · answered Nov 09 '19 at 13:33

A shard key in MongoDB maps to a partition key in Cosmos DB. When you create your collection, you're asked to specify a partition key, if you create it via the portal. Or you can specify the shard key via MongoDB's command line tool or via code (where it's then called a shard key). That shard key must exist in your document.

In your case, your document only contains id and name (and you can use either of those as your shard key). Alternatively, you can specify an additional property as your shard key. For example, maybe schoolid:

student_data = {
  "id": "abc123",
  "name": "jack",
  "schoolid": "school1" 
}

When creating your collection, you'd specify schoolid as your partition key. At this point, your document-save would work, as you will have a shard key (partition key) included in your document.

In your original example, you specified school.students as your shard key. That would require you having a property called school with a sub-property called students (which I'm guessing doesn't make too much sense to do):

student_data = {
  "id": "abc123",
  "name": "jack",
  "school": {
     "students": ...
  }
}

score 1 · Answer 2 · answered Nov 09 '19 at 06:53

1

You can create the collection via the mongo shell

db.runCommand( { shardCollection: "myDb.students", key: { id: "hashed" } } )

answered Nov 09 '19 at 06:53

Sajeetharan

216,225
63
350
396

I already have a collection from the Azure UI, as `students` and it asked me to create a shard key while creating the collection. But how should I use the shard key? – Akshay Nov 09 '19 at 07:29
You need to create it from the shell, not from the azure ui – Sajeetharan Nov 09 '19 at 07:59
@Sajeetharan - this isn't true. You can absolutely create a collection via the portal (the shard key is called a *partition key* in the portal). The OP's issue is that they specified a partition key (shard key) that doesn't exist in the document they tried inserting. I explained this fully in my answer. – David Makogon Nov 09 '19 at 15:25
@DavidMakogon Yes, I agree with your point. What I mentioned here was different. As I remember there was a similar issue earlier when you create from the portal even after including the partition key with MongoDB it does not actually work in the manner as expected. I will try to find the post and paste here, even though I have not tried it – Sajeetharan Nov 09 '19 at 15:41
@Sajeetharan - it's possible there could've been an issue in the past. Before posting my answer, I created a new collection, with partition key, and had no issue connecting via mongo shell and writing/reading docs. – David Makogon Nov 09 '19 at 15:50
Thanks for your answer @Sajeetharan but DavidMakogon answer was what I wanted. +1 for the insight – Akshay Nov 10 '19 at 01:38

Shard key and Azure CosmosDB for MongoDB API

2 Answers2