Is there a way to rebuild Dexie keys?

Question

I had a working app that uses Dexie. After upgrading to iOS 10.3, lookups by key are not working. (This is actually an indexeddb problem, not Dexie per se, I'm sure.) I'm still in shock but I have been able to confirm that the data is there by doing db.table.each(function(p) {}, and the fields used in keys are there and correct. But if I do db.table.get(primarykey, function(p) {} or db.table.where("somekey").equals(nonprimarykey).first(function(p) {} p is undefined.

I tried doing .db.table.each and then putting each retrieved object back to see if that would rebuild the keys, and it worked in Firefox, but doesn't work in Safari or Chrome (still can't retrieve by key).

I also tried specifying a new version with the same key structure and an empty upgrade, and that didn't do anything (but I only tried it in Chrome).

Everything is fine if the database is created AFTER installing 10.3 but I'm hoping that my customers won't have to delete their databases.

Is there any way to repair this without losing data?

Answer 1

This seems to be an upgrade bug in Safari and should really be filed on bugs.webkit.org. Assume this is something that will be fixed there as the Safari team is very responsive when it comes to critical bugs. Please file it!

As for a workaround, I would suggest to recreate the database. Copy the database to a new database, delete it, then copy back and delete the intermediate copy. I've not verified the code below, so you have to test it.

function check_and_fix_IOS_10_3_upgrade_issue () {
    return (somehowCheckIfWeNeedToDoThis) ?
        recreateDatabase() : Promise.resolve();
}

function recreateDatabase () {
    copyDatabase("dbName", "dbName_tmp").then(()=>{
        return Dexie.delete("dbName");
    }).then(()=>{
        return copyDatabase("dbName_tmp", "dbName");
    }).then(()=>{
        return Dexie.delete("dbName_tmp");
    });
}

function copyDatabase(fromDbName, toDbName) {
    return new Dexie(fromDbName).open().then(db => {
        let schema = db.tables.reduce((schema, table) => {
            schema[table.name] = [table.schema.primKey.src]
                .concat(table.schema.indexes.map(idx => idx.src))
                .join(',');
        }, {});

        let dbCopy = new Dexie(toDbName);
        dbCopy.version(db.verno).stores(schema);

        return dbCopy.open().then(()=>{
            // dbCopy is now successfully created with same version and schema as source db.
            // Now also copy the data
            return Promise.all(
                db.tables.map(table =>
                    table.toArray().then(rows => dbCopy.table(table.name).bulkAdd(rows))));
        }).finally(()=>{
             db.close();
             dbCopy.close();
        });
    })
}

Regarding "somehowCheckIfWeNeedToDoThis", I can't answer exactly how to do it. Maybe user-agent sniff + cookie (set persistent cookie when fixed, so that it wont be recreated over and over). Maybe you'll find a better solution.

Then before you open your database (maybe before your app is launched) you'd need to do something like:

check_and_fix_IOS_10_3_upgrade_issue()
    .then(()=>app.start())
    .catch(err => {
        // Display error, to user
    });

Answer 2

I ran into the same issue, using the db.js library. All of my app data is wiped on upgrade.

Based on my tests, it looks like the 10.2 -> 10.3 upgrade is wiping any data in tables that have autoIncrement set to false. Data saved in autoIncrement=true tables is still accessible after the upgrade.

If this is the case, it's a pretty serious bug. The autoIncrement function of Safari had a host of troubles and caused a lot of us to switch to managing our own IDs instead.

I haven't tested this theory with vanilla JS yet. if someone wants to do that please add your results to the bugs.webkit.org ticket