14

I'm developing an RPG in Dart, and I'm going to use IndexedDB for data persistence.

I will have two databases: one for read-only access and one for read-write access where save games will be stored. I was just wondering if I should read required data directly from the database or cache it in Maps. I could potentially have several hundred records that need to be pulled from the read-only database (enemies, game maps etc.) and I though pulling everything from the database may be less efficient than using Dart's Maps.

Oh, also each database will be stored in a map. Object Stores will be nested maps inside that map.

Should I read directly from the database, or should I put everything into a Map and read from there?

EDIT: Forgot to mention, the read-only database will be initialised with data from a JSON file located on the user's machine, not through AJAX.

Melvin Sowah
  • 700
  • 1
  • 10
  • 29

2 Answers2

18

I am confident that hundreds of records will present you no issue in IndexedDB. IDB was designed with that kind of scale in mind, and its async APIs -- while vexing for novices -- make sure your app stays responsive by design.

I am working on a demo designed to push IDB further than it should go, and have some easy-to-reach statistics for you. These are gets on a single index in a single store on a database.

Gets are blazing fast in IndexedDB. The issue with IDB at scale is typically writes.

One thousand success callbacks, one complete callback, were sub-second:

enter image description here

Ten thousand success callbacks, one complete callback, was about 5 seconds:

enter image description here

More than fifty thousand success callbacks fired in less than a minute:

enter image description here

Writes are much slower - bursty at first, but then slow after minutes and dog slow after hours. That's with any schema, but you'd likely have multiple indexes on location (both latitude and longitude at least, I imagine) so your writes will be especially slow (more indexes, more work to do to main those in inserts and updates).

Layout for the stats above (just as important as the stats themselves, make sure to design your schema according to how you need to access it):

enter image description here enter image description here enter image description here

buley
  • 28,032
  • 17
  • 85
  • 106
  • 8
    I wouldn't call 1k reads/sec blazingly fast. This is 1/10000 of what a hash table in C# would offer. JavaScript maps probably offer performance that is somewhere in between, but orders of magnitude more than IDB. – usr Mar 22 '14 at 16:55
  • 3
    Good point that this opinion does depends that against which you compare these numbers. For me, when you compare this local access to network transfer - opening up an HTTP connection and firing packets across the planet - "blazing fast" is not a mischaracterization. Comparing against C# - perhaps you're right. – buley Mar 22 '14 at 16:57
  • How would loading everything from a local JSON file into the database perform? Or does local copy vs. AJAXed copy not make a difference – Melvin Sowah Mar 22 '14 at 17:14
  • Yeah, I wouldn't call this exactly blazing fast either. A Redis database will walk circles around this, *over the network*. For a local database, this is really, really slow. – JulianR Mar 22 '14 at 18:30
  • Guilmon if you've already read the file into memory,you may not need IDB. But same constraints would apply. @JulianR also good points but one benefit is that you don't need to maintain such a server (or an internet connection to it). – buley Mar 22 '14 at 18:49
  • I should note this is using my own wrapper library. "Native" code optimized for the given approach will always run faster than my generalized framework API. – buley Mar 22 '14 at 18:51
  • 1
    @editor - Yeah, I wasn't suggesting Redis as an alternative to IndexedDB, they're not interchangable, but if Redis can be so fast over the network, why is IndexedDB so slow on my local hard drive? – JulianR Mar 22 '14 at 19:24
  • 1
    I'm an IDB fanboy, but I won't mistake: my numbers are via IDB middleware and middleware is slow. Just the act of observing i/o for stats purposes is enough to slow everything down. To get numbers, I have a plugin that observes input and output to calculate throughput (meaning extra function calls each callback). When we're taking about hundreds or thousands per second, that alone is enough to skew results significantly. – buley Mar 22 '14 at 19:47
  • 1
    It also depends how you are writing to your IDB. If you are doing everything in a single transaction it will be faster then a transaction for every write. Also it depends a lot on the browser. My experince is that IE is the fastest, but I see the other browsers are closing the gap. Yesterday I was also looking at batch insert performance. 50k records are now added in +/- 10sec. – Kristof Degrave Mar 23 '14 at 07:01
  • 1
    I just made a benchmark that added 1 000 objects with just one string property of 10 characters in 10 seconds = 10 ms per simple object. One insert per one transaction. – lisak Dec 01 '14 at 20:57
0

I would go with direct database access and then monitor the performance and then optimize where noteable gains are to be expected. Premature optimization is seldom a good idea.

Günter Zöchbauer
  • 623,577
  • 216
  • 2,003
  • 1,567