iOS NSURLCache making up HTTP 404 when going through NSURLProtocol

Question

If you are serious about answering this question, please clone the mini app i have created and see if it misbehaves for you the same way, before speculating about the answer, thank you :)

https://github.com/pavel-zdenek/nsurlprotocol-testbed

A very simple browser: UIWebView with all requests going through bare bones NSURLProtocol reimplementation. It elapses the loading time. The switch un/registers the protocol handler at runtime. Now load one specific site: http://www.rollingstone.com. Besides tons of other resources, the page GETs

http://sjc.ads.nexage.com/js/admax/admax_api.js

which produces an XHR request for

http://sjc.mediation.nexage.net/admax/adMaxApi.do?dcn=<somehash>

PZProtocolHandler OFF: XHR loads instantly. Observed as HTTP 200 on the wire(shark), reflected as successful load in Safari Web Inspector. On further loads, it's not on the wire anymore but still HTTP 200 in Web Inspector. In other words, cached.

PZProtocolHandler ON: Web Inspector reports 3 tries, timing out after 10s each, resulting in HTTP 404. No change with further reloads. Now the interesting part: according to wire(shark), the request goes out and is responded with proper HTTP 200. Furthermore, all 3 tries are sequenced on one and the same TCP channel. "Someone" is making up 404 along the way to UIWebView.

Now, even before reading clever blogs, i found out that iOS URL caching goes thermonuclear when the server response does not contain Cache-Control header. And this particular GET request is such case. But it should not result in 404 failure, should it? Also i have tried to hibernate the default caching through various methods: implementing caching-related delegate calls in NSURLProtocol, creating new request with cache prohibiting flags, using expectedly top knowledge in SDURLCache, even replacing NSURLCache with what i think should be a "null cache". See everything in the project sources.

The 404 persists for me. I doubt that the mere fact of overriding NSURLProtocol would break things so much. I could live without an opportunity to fill a radar for that, thank you. I still hope that i am doing something wrong.

Answer 1

The issue isn't exactly solved, more like narrowed and cleared a little bit. If a new question develops, it will be most probably very different, so i'm self-answering this one.

So it probably isn't a cache issue. The site loads normally when explicit NSOperationQueue is assigned:

_connection = [[NSURLConnection alloc] initWithRequest:request
                                            delegate:self
                                    startImmediately:NO];
// comment out for sure stalling
[_connection setDelegateQueue:[NSOperationQueue mainQueue]];
[_connection start];

I would be tempted to prefer currentQueue, but that's nil at the moment of [NSURLProtocol startLoading]. Which according to Apple doc should mean that startLoading isn't a NSOperation itself, hence there is no queue.

Explicit routing to main thread obviously brings a different performance bottleneck: the delegate calls are stalled when UI is being interacted with, most exemplary when partially loaded page is attempted to be scrolled. While this is also an issue, it's something specific i can fill Apple TSI with. Any new knowledge will go to the github repo.

UPDATE:

Now the issue is fully solved by doing yet another simple thing: create a new vanilla NSOperationQueue and set the NSURLConnectionDelegate queue to it:

[_connection setDelegateQueue:_theNewlyCreatedQueue];

You aren't going to see this in any of the numerous NSURLProtocol howtos on the internets. It would be inappropriately self confident to claim that i know why it improves things so much, but now the offending website loads almost as smoothly as in Mobile Safari (within expected JavaScript execution speed difference) so the problem is solved for me.