2

I have a large NSArray (wordDictionary) and I am creating smaller sub-arrays from it inside a for-loop. If the for-loop is set to 20,000 iterations everything works just fine. But if I increase the for-loop iterations to 200,000 iterations I get a malloc error... Why is that?

I noticed that if I move the sub-array assignment from inside the loop to outside the loop, it solves the problem!(?) Note that all sub-arrays are identical in both cases (this is just to demonstrate the issue). Here is the code with the assignment inside the loop (which causes the malloc error):

    NSArray *subArray;
    //subArray = [wordDictionary subarrayWithRange:(NSRange){50000,20000}];
    for (int i=0;i<200000;i++)
    {
       subArray = [wordDictionary subarrayWithRange:(NSRange){50000,20000}];
       testBool = [subArray containsObject:@"hello"];
    }
    NSLog(@"Done");

The code above works if the sub-array assignment is moved outside the loop (as shown by the commented line)

In the error message I get the following is included:

* mach_vm_map(size=8388608) failed (error code=3) error: can't allocate region Terminating app due to uncaught exception 'NSMallocException' reason: '* NSAllocateObject(): attempt to allocate object of class '__NSArrayI' failed' libc++abi.dylib: terminating with uncaught exception of type NSException

Any hints as to what could be causing this and how to fix it are welcome!! Thanks!!

  • Are you running out of memory? An array with size `20000` seems to be quite big. – Georg Schölly May 03 '14 at 14:06
  • The wordDictionary (NSArray) has close to 200,000 elements in it. And in this example the subArray has 20,000 elements. But there is only one instance and it is re-used over and over so I think it should be fine. However, when the assignment is outside the loop things work fine. Thus, if there is a memory leak in my assignment of the subArray and it is called 200K times, that could well be the reason... – user3582742 May 03 '14 at 14:13
  • Because you don't have an @autorelease block in the loop. – Hot Licks May 03 '14 at 14:19

2 Answers2

3

My guess is that you run out of memory. Every time you call -subarrayWithRange you allocate some memory (depends on the implementation, but could be 20000 * something).

Modern Objective-C uses automatic reference counting instead of garbage collection. That means memory does not get freed instantly, even though you assign another object to subArray.

Try moving it inside a local autorelease pool:

for (int i=0;i<200000;i++)
{
    @autoreleasepool {
        // your code
    }
}
Georg Schölly
  • 124,188
  • 49
  • 220
  • 267
  • 1
    Agreed. Actually this (weird) syntax also works: `for (...) @autoreleasepool { ... }`. – Droppy May 03 '14 at 14:15
  • 2
    @Droppy: There is nothing weird about that syntax. `@autoreleasepool { ... }` is a *statement*, which is exactly what the body of the for-statement should be. – Martin R May 03 '14 at 14:19
  • Well I think it looks weird and is not commonly used, so I mentioned it. – Droppy May 03 '14 at 14:20
  • @autoreleasepool is commonly used by people who don't want to run out of memory. – gnasher729 May 03 '14 at 14:26
1

Even if you follow the advice given and create an autoreleasepool, this is awfully inefficient. You extract a range of 20,000 objects each time, which means 20,000 objects will be retained and later released, for no good reason at all. It's so easily avoided:

for (int i=0;i<200000;i++)
{
   NSRange range = NSMakeRange (50000,20000);
   testBool = [wordDictionary indexOfObject:@"hello" inRange:range] != NSNotFound;
}
NSLog(@"Done");

You might also consider whether an NSSet or NSOrderedSet wouldn't be the right data structure.

gnasher729
  • 51,477
  • 5
  • 75
  • 98
  • You guys are fantastic! I did actually notice a small hit in performance of about 20% in my code with this new call to clear up the memory. It is not that much but I get your point. Your suggestion of not using a subArray at all in this case is great. I will have to digest it and run some tests and see how much I can speed up things by doing it your way instead. Thank you! – user3582742 May 03 '14 at 15:08
  • FYI: In my case I have a list of 200K words that are sorted. I then have a table with 26 indexes where the a-z letters start. If I search the entire list for a word it takes me about 3ms per search. But if I use my index table I can get the time down to 0.5ms (with autorealesepool). But I am still not using the fact that the words are ordered so there must be more I can do to speed things up further. Not using the subArray and autorelease pool will be something I will have to explore further. I will also look into the NSOrderedSet suggestion. Thanks again! – user3582742 May 03 '14 at 15:18
  • @user3582742: What you want [is a binary search](http://en.wikipedia.org/wiki/Binary_search_algorithm). Here's a [stackoverflow question on how to use a binary search on an array](http://stackoverflow.com/questions/11198896/how-to-perform-binary-search-on-nsarray). – Georg Schölly May 03 '14 at 16:59