The Situation: I have a simple word game that uses an English language dictionary (the language dictionary) to provide definitions (in a dictionary data structure, the code-dictionary). Apple's APIs won't do what I need and I could not find a suitable 3rd party solution. The game has worked fine with a makeshift dictionary that was missing words and definitions. I took the Wiktionary database and parsed it way down but hit diminishing returns around 46MB. The dictionary is a simple text file, one word's definition(s) per line. There's also a separate wordlist file with one word per file. I do the parsing with a Python script so I can adjust the file(s) and formats.
The Problem: Reading the language dictionary to a code dictionary takes too long to do it on load. Making the language-dictionary a static const data structure causes XCode to use all my real memory and equivalent swap space eventually rendering my system unusable at around 15GB of swap. This happens whether I attempt a build or not and seems to be XCode scanning it for auto-completion.
The Question: Is there some way to work around this situation? Some kind of directive to not scan the static dictionary file? A simple precompiled library that won't be scanned? An alternative data-structure?
What I've tried: I did a bunch of profiling and sticking in print statements to determine that the language-dictionary loading was the slow part. The code was a simple get next line, add to code-dictionary loop. The static dictionary was my best attempt to fix the loading time so far.
What I haven't tried: A static array and splitting up the dictionary both require almost complete rewrites. I'm not against massive rewrites, if they're likely to resolve the problem. I looked at a plist, but it appears I would need to read that in as well but it would be much larger because of the XML. I considered learning Core Data, but it seemed overkill for a basic, if large lookup table.
