Communication between iOS Broadcast Upload Extension, Frameworks and App

Question

I have a sample app with ViewController.swift and Broadcast Upload app Extension. Inside my project, I have two frameworks that I've created :

1. MySDK, is a swift framework using to analyze and process CMSampleBuffer in order to avoid that Broadcast App Extension consumes too much memory. This SDK has a singleton, a variable isReady, functions func initialize and func analyzeSampleBuffer(_ sampleBuffer: CMSampleBuffer)
2. Broadcaster, is a swift framework using to call MySDK call for analyze and process CMSampleBuffer (yes, sounds like a duplicate with MySDK, but I can't do otherwise, I need these two frameworks). This SDK has a singleton and following functions : func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?), func processSampleBuffer(_ sampleBuffer: CMSampleBuffer)

In this project I've added a Broadcast Upload App Extension named BroadcastExtension. In his main file SampleHandler.swift, in processSampleBuffer function I use my Broadcaster SDK to give responsability from app extension to Broadcaster SDK and after to MySDK to stream CMSampleBuffer that I receive from app extension : Broadcaster.shared.processSampleBuffer(sampleBuffer). At the end, MySDK succeeds to analyze and process CMSampleBuffer but my broadcast upload app extension takes too much memory, and crash after X minutes after screen sharing ( 50Mb max for app extension ). How can I use less memory on app extension ?

Here are my files :

MySDK.swift :

import Foundation
import ReplayKit

@objcMembers public class MySDK {
    public static let shared = MySDK()
    public var isReady = false

    public func initialize() {
        // Init SDK
    }

    public func analyzeSampleBuffer(_ sampleBuffer: CMSampleBuffer) {
        // Analyze
    }
}

Broadcaster.swift :

import Foundation
import ReplayKit
import MySDK

@objcMembers public class Broadcaster: NSObject, Codable {
    public static let shared = Broadcaster()

    public func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
        MySDK.shared.isReady = true
    }

    public func processSampleBuffer(_ sampleBuffer: CMSampleBuffer) {
        if MySDK.shared.isReady {
            MySDK.shared.analyzeSampleBuffer(sampleBuffer)
        }
    }
}

SampleHandler.swift :

import ReplayKit
import Broadcaster

class SampleHandler: RPBroadcastSampleHandler {

    override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
        // User has requested to start the broadcast. Setup info from the UI extension can be supplied but optional.
        Broadcaster.shared.broadcastStarted(withSetupInfo: setupInfo)
    }

    override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
        switch sampleBufferType {
        case RPSampleBufferType.video:
            // Handle video sample buffer
            Broadcaster.shared.processSampleBuffer(sampleBuffer)
            break
        case RPSampleBufferType.audioApp:
            break
        case RPSampleBufferType.audioMic:
            break
        @unknown default:
            fatalError("Unknown type of sample buffer")
        }
    }
}

I would like to get same shared Instance used in app, but in app extension. I tried to put group apps between app and app extension, and to use Userdefaults with suiteName corresponding to the group id, to send the shared instance, but when I receive in app extension, the address memory is not the same, that creates another instance of the object (I want a real singleton between app and app extension). I don't know how to save memory on app extension, and how to communicate between the 2 frameworks, app extension and app to use the same singleton on each part of project.

Here is a hierarchy of my project :

Answer 1

but my broadcast upload app extension takes too much memory, and crash after X minutes after screen sharing ( 50Mb max for app extension ). How can I use less memory on app extension ?

One way to avoid heavy computing and 50Mb memory limit in the Broadcast Upload Extension is to use AVAssetWriter in the SampleHandler, and when it finish you just close it. After this, you can use the shared folder to just copy the (video/audio) from AVAssetWriter.

NSURL* url = [[NSFileManager defaultManager] containerURLForSecurityApplicationGroupIdentifier:@"your group name"];

NSString* sharedVideoPath = [NSString stringWithFormat:@"%@/video.mp4", url.path];
NSFileManager* fileManager = [NSFileManager defaultManager];
NSError* error;
if([fileManager moveItemAtPath:PATH_TO_FILE_FROM_AVASSETWRITER toPath:sharedVideoPath error:&error])
{
    NSLog(@"Successfully moved to shared folder!");
}

After this you just have to use NSUserDefaults to tell the main application the sharedVideoPath and the Main Application will be able to access it.

Answer 2

If you only have to save the screen share video into Files, It is easy. You should have to make a communication between App Extension and a Framework that saves buffers to shared container path(App Groups) using AVAssetWriter. When the broadcast starts and sends buffers on Sample Handler, just pass the buffers into your AVAssetWriter after that when the broadcast stops use finishWriting the AVAssetWriter to finish writing. If you want to know about the video status or process in BroadCastExtension you can use a log manager that writes logs into Shared Container. For example :

    func append(_ sample: CMSampleBuffer, with bufferType: RPSampleBufferType) -> Bool {
    
    guard self.state == .recording else{return false}
    
    guard assetWriter != nil else{return false}
    
    guard sample.isReady else {
        LogManager.shared.e(self,"Buffer Data Is not Ready")
        return true
    }
    
    LogManager.shared.i(self,"Assets Writer Status : \(assetWriter.status.description)")
    
    switch assetWriter.status {
    case .failed:
        LogManager.shared.e(self,"Error occured, status = \(assetWriter.status), \(assetWriter.error!.localizedDescription) \(String(describing: assetWriter.error))")
        return false
    default:
        break
    }
    
    switch bufferType{
    
    case .video:
        self.lastVideoTime = sample.time
        if let lastSampleBuffer = self.lastSampleBuffer {
            videoInput.appendIfPossible(lastSampleBuffer.with(updated: sample.time))
        }
    case .audioMic:
        if self.audioEnabled{
            micInput.appendIfPossible(lastVideoTime != nil ? sample.with(updated: lastVideoTime) : sample)
        }
        
    case .audioApp:
        if self.audioEnabled{
            audioInput.appendIfPossible(lastVideoTime != nil ? sample.with(updated: lastVideoTime) : sample)
        }
    @unknown default:
        LogManager.shared.e(self,"Unkown buffer type")
    }
    return true
}