How to fetch all changes since latest CKServerChangeToken?

Question

This is how I define fetching changes:

func fetchAllChanges(isFetchedFirstTime: Bool) {

    let zone = CKRecordZone(zoneName: "fieldservice")
    let options = CKFetchRecordZoneChangesOperation.ZoneConfiguration()
    
    options.previousServerChangeToken = Token.privateZoneServerChangeToken //initially it is nil        
    let operation = CKFetchRecordZoneChangesOperation(recordZoneIDs: [zone.zoneID], configurationsByRecordZoneID: [zone.zoneID: options])
    operation.fetchAllChanges = isFetchedFirstTime
    operation.database = CloudAssistant.shared.privateDatabase
    // another stuff
}

When I fetch all of them first time, then fetchAllChanges is false. So I only get server change token and save it for another use. No changes for records is returned. And it is ok;)

The problem is when I try to fetch it SECOND TIME. Since then nothing changed, server change token is not nil now, but fetchAllChanges is true because I need all the changes since first fetch (last server change token). It should work like this in my opinion.

But the SECOND TIME I got ALL THE CHANGES from my cloudkit (a few thousands of records and alll the changes). Why? I thought I told cloudkit that I do not want it like this. What am I doing wrong?

I have implemented @vadian answer, but my allChanges is always empty. Why?

func fetchPrivateLatestChanges(handler: ProgressHandler?) async throws -> ([CKRecord], [CKRecord.ID]) {
    /// `recordZoneChanges` can return multiple consecutive changesets before completing, so
    /// we use a loop to process multiple results if needed, indicated by the `moreComing` flag.
    var awaitingChanges = true
    
    var changedRecords = [CKRecord]()
    var deletedRecordIDs = [CKRecord.ID]()
    let zone = CKRecordZone(zoneName: "fieldservice")
    while awaitingChanges {
        /// Fetch changeset for the last known change token.
        print("TOKEN: - \(lastChangeToken)")
        let allChanges = try await privateDatabase.recordZoneChanges(inZoneWith: zone.zoneID, since: lastChangeToken)
        
        /// Convert changes to `CKRecord` objects and deleted IDs.
        let changes = allChanges.modificationResultsByID.compactMapValues { try? $0.get().record }
        print(changes.count)
        changes.forEach { _, record in
            print(record.recordType)
            changedRecords.append(record)
            handler?("Fetching \(changedRecords.count) private records.")
        }
        
        let deletetions = allChanges.deletions.map { $0.recordID }
        deletedRecordIDs.append(contentsOf: deletetions)
        
        /// Save our new change token representing this point in time.
        lastChangeToken = allChanges.changeToken
        
        /// If there are more changes coming, we need to repeat this process with the new token.
        /// This is indicated by the returned changeset `moreComing` flag.
        awaitingChanges = allChanges.moreComing
    }
    return (changedRecords, deletedRecordIDs)
}

And here is what is repeated on console:

TOKEN: - nil

0

TOKEN: - Optional(<CKServerChangeToken: 0x1752a630; data=AQAAAAAAAACXf/////////+L6xlFzHtNX6UXeP5kslOE>)

0

TOKEN: - Optional(<CKServerChangeToken: 0x176432f0; data=AQAAAAAAAAEtf/////////+L6xlFzHtNX6UXeP5kslOE>)

0

TOKEN: - Optional(<CKServerChangeToken: 0x176dccc0; data=AQAAAAAAAAHDf/////////+L6xlFzHtNX6UXeP5kslOE>)

0

... ...

This is how I use it:

TabView {
    //my tabs
}
.tabViewStyle(PageTabViewStyle())
.task {
    await loadData()
}

private func loadData() async {
    await fetchAllInitialDataIfNeeded { error in
        print("FINITO>>")
        print(error)
    }
}

private func fetchAllInitialDataIfNeeded(completion: @escaping ErrorHandler) async {
    isLoading = true
    do {
        let sthToDo = try await assistant.fetchPrivateLatestChanges { info in
            self.loadingText = info
        }
        print(sthToDo)
    } catch let error as NSError {
        print(error.localizedDescription)
    }

vadian · Accepted Answer · 2023-01-22T16:43:15.977

2

Assuming you have implemented also the callbacks of CKFetchRecordZoneChangesOperation you must save the token received by the callbacks permanently for example in UserDefaults.

A smart way to do that is a computed property

var lastChangeToken: CKServerChangeToken? {
    get {
        guard let tokenData = UserDefaults.standard.data(forKey: Key.zoneChangeToken) else { return nil }
        return try? NSKeyedUnarchiver.unarchivedObject(ofClass: CKServerChangeToken.self, from: tokenData)
    }
    set {
        if let token = newValue {
            let tokenData = try! NSKeyedArchiver.archivedData(withRootObject: token, requiringSecureCoding: true)
            UserDefaults.standard.set(tokenData, forKey: Key.zoneChangeToken)
        } else {
            UserDefaults.standard.removeObject(forKey: Key.zoneChangeToken)
        }
    }
}

The struct Key is for constants, you can add more keys like the private subscription ID etc.

struct Key {
    let zoneChangeToken = "zoneChangeToken"
}

Secondly I highly recommend to use the async/await API to fetch the latest changes because it get's rid of the complicated and tedious callbacks.

As you have a singleton CloudAssistant implement the method there and use a property constant for the zone. In init initialize the privateDatabase and also the zone properties.

This is the async/await version of fetchLatestChanges, it returns the new records and also the deleted record IDs

/// Using the last known change token, retrieve changes on the zone since the last time we pulled from iCloud.
func fetchLatestChanges() async throws -> ([CKRecord], [CKRecord.ID]) {
    /// `recordZoneChanges` can return multiple consecutive changesets before completing, so
    /// we use a loop to process multiple results if needed, indicated by the `moreComing` flag.
    var awaitingChanges = true
    
    var changedRecords = [CKRecord]()
    var deletedRecordIDs = [CKRecord.ID]()
    
    while awaitingChanges {
        /// Fetch changeset for the last known change token.
        let allChanges = try await privateDatabase.recordZoneChanges(inZoneWith: zone, since: lastChangeToken)
        
        /// Convert changes to `CKRecord` objects and deleted IDs.
        let changes = allChanges.modificationResultsByID.compactMapValues { try? $0.get().record }
        changes.forEach { _, record in
            changedRecords.append(record)
        }
        
        let deletetions = allChanges.deletions.map { $0.recordID }
        deletedRecordIDs.append(contentsOf: deletetions)
        
        /// Save our new change token representing this point in time.
        lastChangeToken = allChanges.changeToken
        
        /// If there are more changes coming, we need to repeat this process with the new token.
        /// This is indicated by the returned changeset `moreComing` flag.
        awaitingChanges = allChanges.moreComing
    }
    return (changedRecords, deletedRecordIDs)
}

edited Jan 22 '23 at 16:43

answered Jan 21 '23 at 17:58

vadian

274,689
30
353
361

Wow, it looks very smart again. I will try to study it tomorrow and implement all of it. I am really impressed. It looks like the simplest way to fetch all changes. – Bartłomiej Semańczyk Jan 21 '23 at 23:24
I am trying to implement it, but I do have one little problem:. I am trying to call it and fetch all within: `.onAppear(perform: loadData)`. And the issue is `Invalid conversion from 'async' function of type '() async -> ()' to synchronous function type '() -> Void'`. Can it still be async in watchos? – Bartłomiej Semańczyk Jan 22 '23 at 09:18
Replace `.onAppear` with `.task` or add en explicit `Task {}` block. – vadian Jan 22 '23 at 09:19
But will it be called on appear then? – Bartłomiej Semańczyk Jan 22 '23 at 09:22
Yes, it does, check it out. – vadian Jan 22 '23 at 09:23
I updated my question with your solution. This is temporary solution to check if it works, but it doesnt. Could you look at that? It returns always no changes. But it should return over 2k records since it is initial fetch. – Bartłomiej Semańczyk Jan 22 '23 at 09:50
You don't use my suggestion to save the token in `UserDefaults`, your token (`temporaryToken`) is being thrown away. – vadian Jan 22 '23 at 10:05
Ohh. I thought it doesnt matter. My first intention was to save final token after I successfuly save it to core data. Ok, let me try. My apologize. But in this case I have one more question. Because I save it to core data after ALL changesets are fetched, what happens if it fail while saving? Do I have to then reset and fetch all again or only from some point? I mean the case from future when first fetch was long time ago. – Bartłomiej Semańczyk Jan 22 '23 at 10:10
Please re-read the bold part in the first paragraph of the answer, this applies also for `async/await`. Regarding the error handling it depends on the concrete CoreData implementation. But why don't you use `NSPersistentCloudKitContainer`? It does the job to sync data between CloudKit and Core Data seamlessly. – vadian Jan 22 '23 at 10:14
I have implemented everything exactly like you said, but it still returns no changes at all... I think the case is not here where I save token, userdefaults or not... but somewhere else. I have updated the question. – Bartłomiej Semańczyk Jan 22 '23 at 15:29
Something is wrong because it should fetch all lifetime changes when I pass nil as initial token, shouldnt it? – Bartłomiej Semańczyk Jan 22 '23 at 15:44
Please do not edit my answer. The code works. I use it myself. The problem is likely related to your odd handling of the token. Actually you should not touch it. In case of an error the error is thrown and the token is not being updated. – vadian Jan 22 '23 at 16:46
Oh, sorry, It was by mistake. I was wondering why my question didnt change after edit. Look at the code. I will add code how I use it. Please tell me more if there is anything suspicious. I just copy and paste the code for `var lastChangeToken: CKServerChangeToken?` except key which is here `PrivateZoneServerChangeTokenKey` – Bartłomiej Semańczyk Jan 22 '23 at 16:58
Ok, it works... first ~60 changesets was empty because there was no changes. Another ones, 70 change set and above was with 6, 11, 2, 45 changes. Now I understand that it returns not everything at once separated into different packs. But every pack is a kind of changes in short part of database lifetime. Am I right? – Bartłomiej Semańczyk Jan 22 '23 at 19:10

score 1 · Answer 2 · answered Jan 21 '23 at 16:29

I believe you misunderstand how this works. The whole point of passing a token to the CKFetchRecordZoneChangesOperation is so that you only get the changes that have occurred since that token was set. If you pass nil then you get changes starting from the beginning of the lifetime of the record zone.

The fetchAllChanges property is very different from the token. This property specifies whether you need to keep calling a new CKFetchRecordZoneChangesOperation to get all of the changes since the given token or whether the framework does it for you.

On a fresh install of the app you would want to pass nil for the token. Leave the fetchAllChanges set to its default of true. When the operation runs you will get every change ever made to the record zone. Use the various completion blocks to handle those changes. In the end you will get an updated token that you need to save.

The second time you run the operation you use the last token you obtained from the previous run of the operation. You still leave fetchAllChanges set to true. You will now get only the changes that may have occurred since the last time you ran the operation.

The documentation for CKFetchRecordZoneChangesOperation shows example code covering all of this.

Ok, thank you for your answer, but the issue is that I do not want all the changes for the lifetime. I need ONLY the changes since TODAY (at the time when I fresh install the app, in this case on Apple Watch). You know how can I achieve that? — Bartłomiej Semańczyk, Jan 21 '23 at 23:19
`CKFetchRecordZoneChangesOperation` doesn't work that way. You can only get the changes since whatever token you provide. Perhaps you can use `CKQuery`/`CKQueryOperation` to query records with a certain date range. — HangarRash, Jan 21 '23 at 23:37

How to fetch all changes since latest CKServerChangeToken?

2 Answers2