17

Loading data and store them in indexeddb database. Periodically I have the database crashes and lost access to it. Give me, please, a solution how use indexeddb asynchronously!

Sample code that I'm use now:

var dataTotal = 0;
var threads = 6;

//openIndexeddbConnection();

function start(total){

dataTotal = total;
  for (var i = 0; i < threads; i++) {
    loadData(i);
  }
}

function loadData(dataNum){
  var dataNext = dataNum + threads;
  if(dataNext > dataTotal){
    //checkEnd();
    return;
  }

  $.ajax({
    url: baseUrl,
    data: {offset: dataNum},
    success: function (data) {
      successData(dataNext, data);
    },
    type: 'GET'
  });
}

function successData(dataNext, data){
  var dataArray = data.split(';');

  saveData(dataArray);

  loadData(dataNext);
}

function saveData(dataArray){

  putItem();
  function putItem(i) {
    var count = i || 0;
    if(dataArray.length <= i){
      return;
    }

    var transaction = Indexeddb.transaction([dataTableName], "readwrite");
    transaction.onsuccess = function (event) {
      //continue
      putItem(count);
    };
    var objectStore = transaction.objectStore(dataTableName);

    var request = objectStore.add({data: dataArray[count++]});
  }
}
Alexander Abakumov
  • 13,617
  • 16
  • 88
  • 129
Greg
  • 171
  • 1
  • 1
  • 5

2 Answers2

19

You can use promises to load and save data asynchronously to indexedDB. Here are two example functions to load and save data to a simple objectstore in indexedDB.

Asynchronous loading from indexedDB:

function loadFromIndexedDB(storeName, id){
  return new Promise(
    function(resolve, reject) {
      var dbRequest = indexedDB.open(storeName);

      dbRequest.onerror = function(event) {
        reject(Error("Error text"));
      };

      dbRequest.onupgradeneeded = function(event) {
        // Objectstore does not exist. Nothing to load
        event.target.transaction.abort();
        reject(Error('Not found'));
      };

      dbRequest.onsuccess = function(event) {
        var database      = event.target.result;
        var transaction   = database.transaction([storeName]);
        var objectStore   = transaction.objectStore(storeName);
        var objectRequest = objectStore.get(id);

        objectRequest.onerror = function(event) {
          reject(Error('Error text'));
        };

        objectRequest.onsuccess = function(event) {
          if (objectRequest.result) resolve(objectRequest.result);
          else reject(Error('object not found'));
        };
      };
    }
  );
}

Asynchronous saving to indexedDB:

function saveToIndexedDB(storeName, object){
  return new Promise(
    function(resolve, reject) {
      if (object.id === undefined) reject(Error('object has no id.'));
      var dbRequest = indexedDB.open(storeName);

      dbRequest.onerror = function(event) {
        reject(Error("IndexedDB database error"));
      };

      dbRequest.onupgradeneeded = function(event) {
        var database    = event.target.result;
        var objectStore = database.createObjectStore(storeName, {keyPath: "id"});
      };

      dbRequest.onsuccess = function(event) {
        var database      = event.target.result;
        var transaction   = database.transaction([storeName], 'readwrite');
        var objectStore   = transaction.objectStore(storeName);
        var objectRequest = objectStore.put(object); // Overwrite if exists

        objectRequest.onerror = function(event) {
          reject(Error('Error text'));
        };

        objectRequest.onsuccess = function(event) {
          resolve('Data saved OK');
        };
      };
    }
  );
}

Example usage code

var data = {'id' : 1, 'name' : 'bla'};

saveToIndexedDB('objectstoreName', data).then(function (response) {
  alert('data saved');
}).catch(function (error) {
  alert(error.message);
});

// Load some data
var id = 1;
loadFromIndexedDB('objectstoreName', id ).then(function (reponse) {
  data = reponse;
  alert('data loaded OK');
}).catch(function (error) {
  alert(error.message);
});
Jan Derk
  • 2,552
  • 27
  • 22
  • 1
    Thanks for the answer. I had planned in the future to use clean Promise, or rxjs. With Promise more comfortable catch errors. But I want to make code without any errors. In addition, in the example you open every time a new indxedDBbut connection, but I use only only one. I think we should use something like Concurrency Pattern: Producer and Consumer – Greg Jan 11 '17 at 20:09
  • Love this answer. This is what I needed – Dohan Smit Jun 23 '21 at 15:47
6

I've been using idb - a simple library that wraps IndexedDB with promises. These make asynchronous DB actions much easier to use.

If you're targeting Chrome (or are using a transpiler that supports them) you can use async and await to simplify your code:

async function saveData(dataArray) {
    const db = await idb.open('YourDB', currentVersion, upgradeFunction);
    const tran = await db.transaction('StoreName', 'readwrite');
    const store = tran.objectStore('StoreName');

    // This will add the items sequentially
    for(let item of dataArray) {
        await store.add({data:item});
    }
}
Keith
  • 150,284
  • 78
  • 298
  • 434
  • 1
    Thanks for your answer. IDB is an interesting library, but I think with IndexedDB I can work without unnecessary wrappers. Also problem of this library that it does not catch the error when IndexedDB blocked. Yes, my target is chrome, but I do't want to use await, my code should work in other browsers too, in addition wait/asynch in beta now, and work slowly. – Greg Jan 14 '17 at 11:48
  • 2
    @Greg - async/await are part of the coming spec, they're no slower than promises as they basically just execute that functionality. With TypeScript you can use them and they'll transpile to something IE can run. I think you can with Babel too, though I haven't done it myself. I'm not sure what you mean about it not catching errors - that's not what it's supposed to do, all it does is wrap callback functions with promises. You error handle in your own code with the result of the promise or by wrapping the await in a regular try catch. – Keith Jan 14 '17 at 12:12
  • Probably you're right about the wait/async, but usually in the begiining new features is not optimized. For example native Promises worked worse than Polifills. Event `blocked` is created when one connection open, and we try to do more. And I think that this event is triggered when conflicting two concurrent transaction. And question, how avoid this error. – Greg Jan 14 '17 at 13:07
  • @Greg promises have been around for a while now - there are some frameworks that use promisfy patterns (rather than `new Promise(func, func)`) and so have fewer closures and will always be quicker than the ES6 spec native ones (but by doing so they fail to meet that spec). I'm not sure that matters though as the difference between them is a tiny factor of the time taken by IO or network access. I'll pay the 1ms overhead each time to not have to download Bluebird :-) – Keith Jan 14 '17 at 13:24
  • For the blocking I haven't found that to be an issue, but I'm opening a new connection for each async action, rather than making multiple concurrent async actions within shared connection. I've not had a problem with opening multiple concurrent connections, so long as they're all the same version. – Keith Jan 14 '17 at 13:27
  • Yes, I had tested adding in loop without waiting callbacks and did't get any errors. But a single connection 1000 times was added for 279 ms, and each time with new connection for 1398 ms. – Greg Jan 14 '17 at 15:09
  • @Greg that suggests that the overhead for opening a connection for each action is around 1-1.3ms. Based on that I'd go for a connection per action by default and switch to a shared connection for bulk actions. – Keith Jan 14 '17 at 15:27