MongoDB Fails Because Key Too Large To Index

Question

I have been attempting to save a MongoDB document, and am getting a strange error concerning the key being too large to index. I have provided an example of a functional document below, as well as one that creates an error:

Functional:

{ uniquenumber: 'Valid Until Addition of Restrictions',
  name: 'Valid Until Addition of Restrictions'
  fieldofstudy: 'Valid Until Addition of Restrictions',
  section: 'Valid Until Addition of Restrictions',
  restrictions:
   { 'Must be assigned one of the following Student Attributes':
      [ 'Non-Res. In-State',
        'Non-Res. Out-of-State',
        'DE Student Non-Res Out-of-St',
        'Resident In-State',
        'Res. Out-of-State' ],
     'Must be enrolled in one of the following Levels': [ 'Graduate' ] },
  times: [],
  professors: [ 'Jo Blo' ] }

Non-Functional:

{
    "uniquenumber": "Valid Until Addition of Restrictions",
    "name": "Valid Until Addition of Restrictions",
    "fieldofstudy": "Valid Until Addition of Restrictions",
    "section": 'Valid Until Addition of Restrictions',
    "restrictions": {
        "May not be enrolled in one of the following Programs": ["MA [LA] Non-thesis option", "MS [AG] Non-thesis option", "MS [AR] Non-thesis option", "MS [BA] Non-thesis option", "MS [COFD] Non-thesis option", "MS [ED] Non-thesis option", "MS [EN] Non-thesis option", "MS [GE] Non-thesis option", "MS [LA] Non-thesis option", "MS [SC] Non-thesis option", "MS [VM] Non-thesis option", "MS NTO (Special Programs)", "MUP [AR] Non-thesis option"],
        "Must be enrolled in one of the following Levels": ["Graduate"],
        "May not be enrolled in one of the following Degrees": ["Master of Agribusiness", "Master of Agriculture", "Master of Architecture", "Master of Business Admin.", "Master of Computer Science", "Master of Education", "Master of Engineering", "Master of Land Econ & Real Est", "Master of Geoscience", "Master of Landscape Arch.", "Master of Public Administratn", "Master of Public Health", "Master of Public Svc & Admin"],
        "May not be enrolled in one of the following Colleges": ["English Language Institute"],
        "Must be assigned one of the following Student Attributes": ["Non-Res. In-State", "Non-Res. Out-of-State", "DE Student Non-Res Out-of-St", "Resident In-State", "Res. Out-of-State"],
        "Must be enrolled in one of the following Classifications": ["G7-Graduate, Master's Level", "G8-Graduate, Doctoral Level", "G9-Graduate, Mas/Doc Admitted"]
    },
    "times": [],
    "professors": []
}

Up until the creation of the restrictions object, the scraper was functioning as expected, so I believe the error can be narrowed down to there. However, it is unclear to me where any key is "too large" to be indexed. If my guess is correct, why is Mongo counting it as a key and how can I remedy the situation?

EDIT: The response to db.Section.getIndexes(); is shown below:

[
        {
                "v" : 2,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "database.Section"
        },
        {
                "v" : 2,
                "unique" : true,
                "key" : {
                        "uniquenumber" : 1
                },
                "name" : "uniquenumber_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "name" : 1
                },
                "name" : "name_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "fieldofstudy" : 1
                },
                "name" : "fieldofstudy_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "course" : 1
                },
                "name" : "course_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "section" : 1
                },
                "name" : "section_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "professors" : 1
                },
                "name" : "professors_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "times" : 1
                },
                "name" : "times_1",
                "ns" : "database.Section",
                "background" : true
        },
        {
                "v" : 2,
                "key" : {
                        "restrictions" : 1
                },
                "name" : "restrictions_1",
                "ns" : "database.Section",
                "background" : true
        }
]

Hi Quontas; can you show us what indexes you have on this collection? — Vince Bowdren, Jun 04 '17 at 21:11
For reference, the total size of an index entry must be less than 1024 bytes (https://docs.mongodb.com/manual/reference/limits/#Index-Key-Limit). The error is indicating that an index exists on this collection such that a value in the non-functional document exceeds that limit. We can't say what is causing the failure without knowing the existing indexes, but I would guess that you have indexed the "restrictions" field. — Adam Harrison, Jun 04 '17 at 22:24
The response to `db.Section.getIndexes();` is shown in the original post. — Quontas, Jun 04 '17 at 22:43
You've got an index on `restrictions` which, based on your data, is completely useless and would be very large. You likely want to drop that index from the collection. — JohnnyHK, Jun 04 '17 at 22:52
Thanks, @JohnnyHK! However, I would like the restrictions to be saved with the document, but maybe not searchable. Is there a proper way of going about that? — Quontas, Jun 04 '17 at 22:54
@JohnnyHK https://stackoverflow.com/questions/44359738/data-in-javascript-object-not-being-saved-to-mongodb — Quontas, Jun 04 '17 at 23:18

score 3 · Accepted Answer · answered Jun 05 '17 at 14:31

3

You've got an index on restrictions which, based on your data, is completely useless and would be very large as that's a sub-document.

You likely want to drop that index from the collection:

db.Section.dropIndex({restrictions: 1});

answered Jun 05 '17 at 14:31

JohnnyHK

305,182
66
621
471

MongoDB Fails Because Key Too Large To Index

1 Answers1