NSURLSession with upload stream - subclassing NSInputStream - com.apple.NSURLConnectionLoader exception

Question

Basic task

I have some multiplatform library which is using some C++ stream interface. I have to use this stream interface to upload data by NSURLSession. My implementation should work on OS X and iOS (currently I'm testing on OS X)

What I did

Task looks quite simple and I was sure I will implement this quite fast. I have configured NSURLSession which is working fine if I'm using NSURLRequest with simple NSData. I'm trying to use stream like this:

        NSURLSessionDataTask *dataTask = [m_Private.session uploadTaskWithStreamedRequest: request];
        HTTPDownoadTaskProxy *dataTaskProxy = [HTTPDownoadTaskProxy new];
        // store data to properly handle delegate
        dataTaskProxy.coreTask = dataTask;
        dataTaskProxy.cppRequest= req;
        dataTaskProxy.cppResponseHandler = handler;
        dataTaskProxy.cppErrorHandler = errorHandler;

        m_Private.streamedDataTasks[dataTask] = dataTaskProxy;

        [dataTask resume];

So far so good. According to documentation of uploadTaskWithStreamedRequest I should receive notification from delegate and I do receive it:

- (void)URLSession: (NSURLSession *)session
              task: (NSURLSessionTask *)task
 needNewBodyStream: (void (^)(NSInputStream *bodyStream))completionHandler
{
    HTTPDownoadTaskProxy *proxyTask = self.streamedDataTasks[task];
    CppInputStreamWrapper *objcInputStream = [[CppInputStreamWrapper alloc] initWithCppInputStream:proxyTask.cppRequest.GetDataStream()];
    completionHandler(objcInputStream);
}

Now I should receive calls in subclass of NSInputStream which is in my case CppInputStreamWrapper, and also it is quite simple:

@implementation CppInputStreamWrapper

- (void)dealloc {
    NSLog(@"%s", __PRETTY_FUNCTION__);
}

- (instancetype)initWithCppInputStream: (const std::tr1::shared_ptr<IInputStream>&) cppInputStream
{
    if (self = [super init]) {
        _cppInputStream = cppInputStream;
    }
    return self;
}

#pragma mark - overrides for NSInputStream
- (NSInteger)read:(uint8_t *)buffer maxLength:(NSUInteger)len {
    return (NSInteger)self.cppInputStream->Read(buffer, len);
}

- (BOOL)getBuffer:(uint8_t **)buffer length:(NSUInteger *)len {
    return NO;
}

- (BOOL)hasBytesAvailable {
    return !self.cppInputStream->IsEOF();
}

#pragma mark - this methods are need to be overridden to make stream working
- (void)scheduleInRunLoop:(__unused NSRunLoop *)aRunLoop
                  forMode:(__unused NSString *)mode
{}

- (void)removeFromRunLoop:(__unused NSRunLoop *)aRunLoop
                  forMode:(__unused NSString *)mode
{}

#pragma mark - Undocumented CFReadStream Bridged Methods
- (void)_scheduleInCFRunLoop:(__unused CFRunLoopRef)aRunLoop
                     forMode:(__unused CFStringRef)aMode
{}

- (void)_unscheduleFromCFRunLoop:(__unused CFRunLoopRef)aRunLoop
                         forMode:(__unused CFStringRef)aMode
{}

- (BOOL)_setCFClientFlags:(__unused CFOptionFlags)inFlags
                 callback:(__unused CFReadStreamClientCallBack)inCallback
                  context:(__unused CFStreamClientContext *)inContext {
    return NO;
}

@end

So I'm using workaround needed when subclassing NSInputStream.

Problem

Now this should work. But I'm not receiving any call of methods of CppInputStreamWrapper (except for my call when construction object).

No errors no warning are reported, nothing!

When I've added exception breakpoint I'm catching

thread #8: tid = 0x155cb3, 0x00007fff8b770743 libobjc.A.dylib`objc_exception_throw, name = 'com.apple.NSURLConnectionLoader', stop reason = breakpoint 1.1

This comes from thread com.apple.NSURLConnectionLoader which I didn't create.

I'm totally puzzled and have no idea what else I can do.

Update

I've used code form link in comment which is hosted on github. Now at least some parts of my class are invoked by framework, but I see strange crash.

Crash is located in this method:

- (BOOL)_setCFClientFlags:(CFOptionFlags)inFlags
                 callback:(CFReadStreamClientCallBack)inCallback
                  context:(CFStreamClientContext *)inContext {

    if (inCallback != NULL) {
        requestedEvents = inFlags;
        copiedCallback = inCallback;
        memcpy(&copiedContext, inContext, sizeof(CFStreamClientContext));

        if (copiedContext.info && copiedContext.retain) {
            copiedContext.retain(copiedContext.info);
        }

        copiedCallback((__bridge CFReadStreamRef)self, kCFStreamEventHasBytesAvailable, &copiedContext); // CRASH HERE
    } else {
        requestedEvents = kCFStreamEventNone;
        copiedCallback = NULL;
        if (copiedContext.info && copiedContext.release) {
            copiedContext.release(copiedContext.info);
        }

        memset(&copiedContext, 0, sizeof(CFStreamClientContext));
    }

    return YES;

}

Crash is EXC_BAD_ACCESS (when running tests on OS X). when I see this code everything looks fine. It should work! self is pointing to proper object with retain count 3 so I have no idea why it is crashing.

Answer 1

Undocumented private bridging API is not the only problem in custom NSInputStream implementation especially in the context of CFNetworking integration. I'd like to recommend to use my POSInputStreamLibrary as basic building block. Instead of implementing a lot of NSInputStream methods and supporting async notifications you should implement much simpler POSBlobInputStreamDataSource interface. At least you can look at POSBlobInputStream to consult what kind of functionality you should implement to support NSInputStream contract completely.

POSInputStreamLibrary is used in the most popular Russian cloud storage service Cloud Mail.Ru and uploads >1M files per day without any crashes.

Good luck and feel free to ask any questions.

Answer 2

I see you do have implementations of the undocumented CFReadStream bridge methods -- that is one of the more common issues. However... note the comment in the NSStream.h header for the NSStream class:

// NSStream is an abstract class encapsulating the common API to NSInputStream and NSOutputStream.
// Subclassers of NSInputStream and NSOutputStream must also implement these methods.

That means you also need to implement -open, -close, -propertyForKey:, -streamStatus, etc. -- every method that is declared on NSStream and NSInputStream, basically. Try calling -open yourself in your code (which NSURLConnection will eventually do) -- you will get the idea since it should crash right there. You will probably need at least some minimal status handling so that -streamStatus does not return NSStreamStatusNotOpen after -open is called, for example. Basically, every concrete subclass needs to implement the entirety of the API. It's not like a normal class cluster where just a couple of core methods need to be overridden -- even the -delegate and -setDelegate: methods must be implemented (the superclass does not have instance variable storage for it, I'm pretty sure).

AFNetworking has an internal AFMultipartBodyStream which has the minimal implementations needed -- you can see that example inside AFURLRequestSerialization.m. Another code example is HSCountingInputStream.