Different behaviour between enumerateObjectsUsingBlock: and for( in )

Question

Give this code

NSMutableArray *array = [NSMutableArray new];
for (int i = 0; i < 10000; i++) {
    [array addObject:@(i)];
}

queue1 = dispatch_queue_create("com.test_enumaration.1", DISPATCH_QUEUE_CONCURRENT);
queue2 = dispatch_queue_create("com.test_enumaration.2", DISPATCH_QUEUE_CONCURRENT);

dispatch_async(queue1, ^{
    int idx = 0;
    for (NSNumber *obj in array) {
        NSLog(@"[%d] %@", idx, obj);
        idx++;
    }
});

double delayInSeconds = 0.3;
dispatch_time_t popTime = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(delayInSeconds * NSEC_PER_SEC));
dispatch_after(popTime, queue2, ^(void){
    [array removeObjectAtIndex:9000];
    NSLog(@"----");
});

I'm expecting that this code crash because at some point the block dispatched on queue2 get executed concurrently to the enumeration and this will trigger the assertion that you cannot mutate an the array while enumerating. Indeed, this is what happens.

The interesting part is when you substitute for ( in ) with enumerateObjectsUsingBlock:

NSMutableArray *array = [NSMutableArray new];
for (int i = 0; i < 10000; i++) {
    [array addObject:@(i)];
}

queue1 = dispatch_queue_create("com.test_enumaration.1", DISPATCH_QUEUE_CONCURRENT);
queue2 = dispatch_queue_create("com.test_enumaration.2", DISPATCH_QUEUE_CONCURRENT);

dispatch_async(queue1, ^{
    [array enumerateObjectsUsingBlock:^(id obj, NSUInteger idx, BOOL *stop) {
        NSLog(@"[%d] %@",idx, obj);
    }];
});

double delayInSeconds = 0.3;
dispatch_time_t popTime = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(delayInSeconds * NSEC_PER_SEC));
dispatch_after(popTime, queue2, ^(void){
    [array removeObjectAtIndex:9000];
    NSLog(@"----");
});

In all my different test the block that remove the object is executed in the middle of the enumeration (I see the print of @"----") and the interesting thing is that the enumeration behave correctly printing [8999] 8999 and then [9000] 9001.

In this case the array is mutated during the enumeration without firing any assertion. Is this an intended behaviour? If yes, why? I'm I missing something?

possible duplicate of [Objective-C enumerateUsingBlock vs fast enumeration?](http://stackoverflow.com/questions/8509662/objective-c-enumerateusingblock-vs-fast-enumeration) — trojanfoe, Mar 19 '14 at 11:51
@trojanfoe Not really, in the linked answer is stated "both methods protect mutable collections from mutation inside the enumeration loop" in my case I'm not trying to change the change the collection inside the enumeration block but i'm mutating this from a different thread. — Luca Bernardi, Mar 19 '14 at 11:54
@LucaBernardi: *Both* variants crash with the "Collection <__NSArrayM: 0x100113d10> was mutated while being enumerated" exception when I run your code. — Martin R, Mar 19 '14 at 12:40
I run the test again and for me the is know crash and I saw logged the "----" in the middle of the enumeration and in fact the output is [8999] 8999 and the [9000] 9001 — Luca Bernardi, Mar 19 '14 at 14:04
I got the same output as described by @LucaBernardi. Indeed, it seems like enumerateObjectsUsingBlock: does not complain when the array is mutated while being enumerated and simply skip the removed object. — andreag, Mar 19 '14 at 20:54

AndrewS · Answer 1 · 2014-05-14T19:25:46.747

0

Since the introduction of fast enumeration, it's become the go-to method for ... making enumeration fast. Most implementations of enumerations, such as for(in) and enumerateObjectsUsingBlock:, will use fast enumeration under the covers.

Fast enumeration will look at how the data is stored. In the case of NSMutableArray, I would guess that the underlying data is stored in several chunks of data; a ten-thousand item array might be implemented as a hundred chunks of 100 items, with each chunk storing its hundred items in contiguous memory. Analysis of some assembly suggests that (at least on some iOS devices) the class is implemented as a single, giant circular buffer. Either way, the enumerated list might contain multiple contiguous blocks of objects. Ultimately, the exact storage mechanism is irrelevant; it's access to the underlying contiguous storage that makes fast enumeration better than the alternative.

Generally, enumeration is supposed to prevent the list from being mutated. You will always see that with the for(in) enumeration. Apparently some implementations of enumerateObjectsUsingBlock: do not robustly guarantee that the list is not mutated during enumeration. I am getting an assertion failure on the devices that I've tried... but it sounds like there are some devices where this protection is broken. I'd guess that the mutation guard used within NSFastEnumerationState is not complete, perhaps only watching a single chunk instead of the entire array.

I'd consider this a bug in enumerateObjectsUsingBlock:.

Further, any code that might generate an exception here is by definition bad code: you'll need to provide a mechanism to prevent your own code from trying to modify an array while another thread is iterating over it.

edited May 14 '14 at 19:25

answered May 14 '14 at 19:04

AndrewS

8,196
5
39
53

1

do you have any resource for your claim that NSMutableArray is storing its data in chunks? [This article](http://ciechanowski.me/blog/2014/03/05/exposing-nsmutablearray/) shows it is a circular buffer. – vikingosegundo May 14 '14 at 19:13
Mike Ash implied that this was the case in https://mikeash.com/pyblog/friday-qa-2010-04-09-comparison-of-objective-c-enumeration-techniques.html. Bartosz, in the article you mention, likewise is making a guess to the actual implementation. That aside, the implementation itself is not open-sourced, and Apple retains the ability to change the implementation on different devices and OS versions. So, no, I don't *know* that's how it's stored. I'll edit the post to reflect that. – AndrewS May 14 '14 at 19:21
Mike Ash's article doesn't make any statement about the internals of NSMutableArray. – vikingosegundo May 14 '14 at 19:37
Indeed, there is no statement; only an implication. Plus, the specific implementation is irrelevant. The point is to introduce to the reader the idea of "multiple chunks of contiguous objects", which I think is a possible source for the bug. – AndrewS May 14 '14 at 19:42
and where is the implication. I cannot find it. if there where multiple chunks within one array, most of the observations of Bartosz wouldn't be possible. – vikingosegundo May 14 '14 at 19:45
Using a circular buffer allows up to two disjoint chunks of contiguous objects... which happen to be joined by a chunk of contiguous null pointers (or unused object storage). As for Mike Ash's article, search for the phrase "If there are multiple contiguous object stores". – AndrewS May 14 '14 at 19:50
NSFastEnumeration protocol can be adapted for custom classes. those could have different stores, Mike does not imply that is true for arrays. – vikingosegundo May 14 '14 at 19:52
I must have mistakenly inferred it. My bad. – AndrewS May 14 '14 at 19:55

Different behaviour between enumerateObjectsUsingBlock: and for( in )

1 Answers1