MongoDB Schema Redesign with existing data

Question

I have around 200 000 Object names and 20 roles and 500 users. Roles are categorized as admin,domain1,domain2,domain3,domain4..domain 20

Some roles (domain1,domain4 ) can only access to 5 object names and some other roles have access to 500-20 000 object names. Admin role have access to all the objects (2 million). So I've to map all the objects under admin role. So my data model would be like below.,

    {_id:1,
    Role_name: "admin"
    access_objects :{name1,name2 ..........name2000000}
   },
   {_id:2,
    Role_name: "domain1"
    access_objects :{name1,name2 ..........name5}
   },
   {_id:3,
    Role_name: "domain3"
    access_objects :{name1,name2 ..........name500}
   }
   ..
   {_id:5,
    Role_name: "domain5"
    access_objects :{name1,name2 ..........name5000}
   }

Here is the mongodb document 16mb limitation issue came. I'm not able to store 2 million object names in embedded document as shown above for admin.

So we did instead of mapping object names into role, the role is mapped to each operation . My current collection design like below.,

{_id:1,
    object_name: "object_name1"
    applicable_roles :{admin,domain1,domain2,domain3,domain4,domain5}
   },
   {_id:2,
    object_name: "object_name2"
    applicable_roles :{admin,domain1,domain5}
   },
   {_id:3,
    object_name: "object_name3"
    applicable_roles :{domain4}
   }
   ..
   {_id:2000000,
    object_name: "object_name2000000"
    applicable_roles :{domain4,domain1}
   }

In this model , We are facing slow query response.Some domains have access only few objects names but need lot of embedded loops to identify the objects.Already we have required indexes on the collection

1. Please drop the "hello" and "regards" from your questions. This is not a cover letter for employment, but a question to your peers. Keep it relevant. 2. Have I not already commented to you about "scary numbers" in your question? I believe I have. 3. Take a good look at your structure proposal in your question and realize it is not valid JSON ( and therefore BSON ) and think again please. — Blakes Seven, Sep 28 '15 at 12:09

score 0 · Accepted Answer · answered Sep 28 '15 at 12:57

Below are a few suggestions that may help.

First, as you demonstrated, map the applicable_roles to the object_name since there will be a much larger quantity of object names than applicable roles.

// Do this:
{
    _id:1,
    object_name: "object_name1"
    applicable_roles :{admin,domain1,domain2,domain3,domain4,domain5}
}
// Instead of:
{
    _id:1,
    role: "admin"
    applicable_objects: {object1, ...., object200000}
}

Second, use an array as the element type for holding your applicable_roles (not an object with the role names as the property keys). As shown below, the applicable_roles are now in an array.

{
    _id:1,
    object_name: "object_name1"
    applicable_roles :[ admin,domain1,domain2,domain3,domain4,domain5 ]
}

(By the way, the applicable_roles and applicable_objects are neither objects nor arrays in your post. Typo?)

Third, take advantage of indexes. By adding an index on the applicable_roles array then you will see improved performance.

db.yourcollection.createIndex({ applicable_roles : 1 })

(If you have your applicable_roles as an object instead of an array then your applicable_roles object schema will be different for each row since, I'm assuming, the object keys are the role names. This format would prevent you from having an entirely useful index on an applicable_roles field since the object schema for that field would not be consistent across rows.)

@whyceewhilt . Yes, Applicable_roles are arrays. Even I have the index on the array system is taking time to fetch the data. Apart from the data models mentioned above is there any other better data models we can consider? — Root, Sep 29 '15 at 06:30

MongoDB Schema Redesign with existing data

1 Answers1