iOS AudioUnit callback hiccup on going to background mode

Question

I'm writing an application to play RTP audio stream on iOS devices. It works well on both foreground and background modes, but while switching to background it kind of stuck for about a second. Trying to investigate that issue, i found what during transition to background mode, time between calls of AudioUnit callback function rises from 5 milliseconds up to 100 milliseconds, and when transitions is finished - falls back to 5 milliseconds. Audio initialization code:

-(BOOL)initOutput {
    OSStatus status;

    AudioStreamBasicDescription audioFormat;

    AudioComponentDescription desc;
    desc.componentType = kAudioUnitType_Output;
    desc.componentSubType = kAudioUnitSubType_RemoteIO;
    desc.componentFlags = 0;
    desc.componentFlagsMask = 0;
    desc.componentManufacturer = kAudioUnitManufacturer_Apple;

    AudioComponent component = AudioComponentFindNext(NULL, &desc);

    status = AudioComponentInstanceNew(component, &audioUnit);
    if (status != 0) {
        NSLog(@"Error creating audio component instance");
        return false;
    }

    audioFormat.mSampleRate         = samplerate; // 48000
    audioFormat.mFormatID           = kAudioFormatLinearPCM;
    audioFormat.mFormatFlags        = kAudioFormatFlagsCanonical;
    audioFormat.mFramesPerPacket    = 1;
    audioFormat.mChannelsPerFrame   = channels; // 2
    audioFormat.mBitsPerChannel     = 16;
    audioFormat.mBytesPerPacket     = channels * 2;
    audioFormat.mBytesPerFrame      = audioFormat.mBytesPerPacket;
    audioFormat.mReserved           = 0;

    status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &audioFormat, sizeof(audioFormat));
    if (status != 0) {
        NSLog(@"Error setting kAudioUnitProperty_StreamFormat %i", status);
        return false;
    }

    AURenderCallbackStruct callbackStruct;
    callbackStruct.inputProc = playbackCallback;
    callbackStruct.inputProcRefCon = (__bridge void*)self;
    status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Global, 0, &callbackStruct, sizeof(callbackStruct));
    if (status != 0) {
        NSLog(@"Error setting kAudioUnitProperty_SetRenderCallback");
        return false;
    }

    status = AudioUnitInitialize(audioUnit);
    if (status != 0) {
        NSLog(@"Error initializing unit");
        return false;
    }

    return true;
}

AVAudioSession category is AVAudioSessionCategoryPlayback. CPU load never goes higher than 10%. What am i doing wrong, and how to fix it?