How to download and being notified, a list of files in iCloud Document, in a scalable way?

Question

I wish to download, and being notified, on files at iCloud Document.

The list of filenames are known up-front as we store the filenames in a iCloudKit (Not iCloud Document)

The list of filenames will be updated in the middle of runtime. Then, once the list of filenames is update, we need to continue to search and download for the files from iCloud Document.

I wish to make good use of NSMetadataQuery and listen to list of notifications like NSMetadataQueryDidStartGathering.

The closest example I can find so far is from https://medium.com/swlh/restore-your-applications-data-from-icloud-and-track-progress-a63a8bd3de38

The example is showing how to handle single file, without resource cleanup (The observers are not removed)

I modified the example to (also single file handling without resource cleanup)

Imperfect example as it can only handle single file, and not performing observers removal

class iCloudDocumentManager {
    static let INSTANCE = iCloudDocumentManager()
    
    var query: NSMetadataQuery!

    private init() {
        initialiseQuery()
        addNotificationObservers()
        
        query.operationQueue?.addOperation({ [weak self] in
            self?.query.start()
            self?.query.enableUpdates()
        })
    }
    
    private func initialiseQuery() {
        query = NSMetadataQuery.init()
        query.operationQueue = .main
        
        query.predicate = NSPredicate(format: "%K == %@", NSMetadataItemFSNameKey, "1ccd0ac7-b6c4-4804-9ee1-1031e57c4ec7.jpeg")
        
        query.searchScopes = [
            NSMetadataQueryUbiquitousDocumentsScope
        ]
    }
    
    private func addNotificationObservers() {
        
        NotificationCenter.default.addObserver(forName: NSNotification.Name.NSMetadataQueryDidStartGathering, object: query, queue: query.operationQueue) { (notification) in
            print(">>>> NSMetadataQueryDidStartGathering")
            self.processCloudFiles()
        }
        
        NotificationCenter.default.addObserver(forName: NSNotification.Name.NSMetadataQueryGatheringProgress, object: query, queue: query.operationQueue) { (notification) in
            print(">>>> NSMetadataQueryGatheringProgress")
            self.processCloudFiles()
        }

        NotificationCenter.default.addObserver(forName: NSNotification.Name.NSMetadataQueryDidUpdate, object: query, queue: query.operationQueue) { (notification) in
            print(">>>> NSMetadataQueryDidUpdate")
            self.processCloudFiles()
        }
    }
    
    @objc func processCloudFiles() {
           
       if query.results.count == 0 { return }

       
       for item in query.results {
           guard let item = item as? NSMetadataItem else { continue }
           guard let fileItemURL = item.value(forAttribute: NSMetadataItemURLKey) as? URL else { continue }
           
           try? FileManager.default.startDownloadingUbiquitousItem(at: fileItemURL)
           
           if let fileDownloaded = item.value(forAttribute: NSMetadataUbiquitousItemDownloadingStatusKey) as? String, fileDownloaded == NSMetadataUbiquitousItemDownloadingStatusCurrent {
               
               query.disableUpdates()
               query.operationQueue?.addOperation({ [weak self] in
                   self?.query.stop()
               })
               
               print(">>>> Download complete \(fileItemURL)")
           
           } else if let error = item.value(forAttribute: NSMetadataUbiquitousItemDownloadingErrorKey) as? NSError {
               print(error.localizedDescription)
           } else {
               if let keyProgress = item.value(forAttribute: NSMetadataUbiquitousItemPercentDownloadedKey) as? Double {
                   print(">>>> File downloaded percent \(fileItemURL) ---", keyProgress)
               }
           }
       }
    }
}

My questions are

When, and how I should remove the observers? After the download is completed? What if the download is failed? How I will be notified so that there is a chance for me to remove the observers?
How I should handle a large amount of files say like 10,000 files? Does that mean I need to have 10,000 of NSMetadataQuery and 3 * 10,000 of observers? What is a scalable way to achieve so?
processCloudFiles will be running in main thread which might cause UI unresponsive. How can I run such operations in background thread?

Thank you.

A scaleable solution could involve fetching all files in a directory or all files that meet a known pattern. 10,000 arbitrary names sounds like it probably won't scale well at all. I've run multiple NSMetaDataQueries simultaneously without issue, but there is sure to be a practical limit. Confirming the files match your list, and only downloading those ones that match would be fairly cheap. — MichaelR, Jun 27 '22 at 00:57
On removing observers. You do this after stopping the query (if you wish - it isn't required these days). — MichaelR, Jun 27 '22 at 01:06

How to download and being notified, a list of files in iCloud Document, in a scalable way?

Imperfect example as it can only handle single file, and not performing observers removal

0 Answers0