20

I'm an embedded guy, not a database guy. I've been asked to redesign an existing system which has bottlenecks in several places.

The embedded device is based around an ARM 9 processor running at 220mHz.

There should be a database of 50k entries (may increase to 250k) each with 1k of data (max 8 filed). That's approximate - I can try to get more precise figures if necessary.

They are currently using SqlLite 2 and planning to move to SqlLite 3.

Without starting a flame war - I am a complete d/b newbie just seeking advice - is that the "best" decision? I realize that this might be a "how long is a piece of string?" question, but any pointers woudl be greatly welcomed. I don't mind doing a lot of reading & research, but just hoped that you could get me off to a flying start. Thanks.

p.s Again, a total rewrite, might not even stick with embedded Linux, but switch to eCos, don't worry too much about one time conversion between d/b formats. Oh, and accesses should be infrequent, at most one every few seconds.

edit: ok, it seems they have 30k entries (may reach 100k or more) of only 5 or 6 fields each, but at least 3 of them can be a search key for a record. They are toying with "having no d/b at all, since the data are so simple", but it seems to me that with multiple keys, we couldn't use fancy stuff like a quicksort() type search (recursive, binary search). Any thoughts on "no d/b", just data-structures?

Btw, one key is 800k - not sure how well SqlLite handles that (maybe with "no d/b" I have to hash that 800k to something smaller?)

Mawg says reinstate Monica
  • 38,334
  • 103
  • 306
  • 551
  • 2
    sqllite seems like a good choice to stick with – Toad Jan 06 '10 at 08:42
  • yeah, i'd recommend sqllite too. – junmats Jan 06 '10 at 08:44
  • 1
    Is the database even your bottleneck? By your numbers, it should be fine unless your storage is really slow. Remember that 10 years ago that would have been considered a fairly fast machine. – Andrew McGregor Jan 06 '10 at 09:23
  • 3
    Wow, that's a sslllooowwww ARM9 processor, running at 0.220 cycles / sec! ;-) (By the way, I like the questions you've asked recently) – Dan Jan 07 '10 at 14:31
  • 220mHz used to be a powerful PC only a few years ago ;-) But the task that the device does won't strain that. Thanks for the compliment on the questions - I'll go check out your answers now ;-) – Mawg says reinstate Monica Jan 08 '10 at 00:50
  • 3
    Mawg: What Dan is coming at, is that you've written mHz instead of MHz. m = milli, M = mega. Semantics :) – Morten Jensen Aug 16 '12 at 08:23

10 Answers10

27

Also SQLite is the Database chosen by virtually all mobile operating systems. Android, Iphone OS and Symbian ship with SQLite which makes me think that manpower was spent to optimize it for the processor in those phones (nearly always ARM).

Valentin
  • 2,066
  • 3
  • 18
  • 14
13

I would stick with SQLite, it's widely supported and pretty rich in features.

selfawaresoup
  • 15,473
  • 7
  • 36
  • 47
6

  • Firebird (previously Interbase) claims to work well embedded.

  • HypersonicQL (HQL) is small and fast and also claims to be suitable for embedded use.

Alas, I have no personal experience to back up either claim.

Carl Smotricz
  • 66,391
  • 18
  • 125
  • 167
4

SQLite is probably a pretty safe bet. However, if performance is really important for your application and you do not need a relational database, I would suggest you take a look at Berkeley DB link text . Berkeley DB is not a relational database though. In other words, if your data is grouped in different tables and you constantly need to query result sets that require relating data from more than one table, you probably need a relational database. Berkeley DB is better suited for something like look up tables (i.e., the data is organized in a few tables and you don't need to query data from more than one of them in order to produce the result sets you want). Berkeley DB is very fast but it will require more work on your end in order to get the most out of it.

figurassa
  • 2,037
  • 2
  • 13
  • 12
  • Just a single table, 5 or 6 fields, but three of them can be used as a unique key... and there are a lot of records - 30k, might become 100k, possibly even 256k – Mawg says reinstate Monica Jan 08 '10 at 00:51
  • 1
    In this case, I'd say Berkeley DB is absolutely perfect for your application. If performance is of utmost importance for your application, I think you should look into Berkeley DB. – figurassa Jan 08 '10 at 01:38
  • Here's a performance comparison of sqlite and Berkeley DB: http://www.sqlite.org/cvstrac/wiki?p=KeyValueDatabase for key-value data storing – Igor Brejc Jan 17 '10 at 20:34
3

if you want an alternative, then berkeleydb is worth looking at. it used to be owned by sleepycat software, but is now available from oracle. it's a barebones database engine; is directly programmable (rather than a sql) frontend. it's used as part of the core engine in many major databases, and as the database in many embedded devices - it used to be particularly popular for managing routing tables in routers. it tends to get overlooked these days for more fashionable setups, but i've found it to be decent, solid and for the numbers you are talking about it can be lightning fast.

James
  • 1,380
  • 11
  • 11
3

8 Years late, but as an update: I've had pretty good experience using Raima Database Manager. If you are looking for a small footprint db, they can get down to 40k. One of the reasons I like RDM is the platform independence, it is portable across 32-bit and 64-bit machines and between big-endian and little-endian architectures as well as support for most operating systems, meaning you can use it on Embedded Linux and eCos as mentioned in the first post. And it's performance gets better as you add better hardware and users as opposed to SQLite

Fredrik S
  • 41
  • 3
  • 1
    Welcome aboard, Frederick. Someone (not me) downvoted you with not explanation. I would ***guess*** it was because you posted a "link only answer" which is frowned upon. We prefer explanations on our site, in case the link dies. – Mawg says reinstate Monica Oct 18 '18 at 13:56
  • Btw, can you give us an idea of how much it costs? – Mawg says reinstate Monica Oct 18 '18 at 13:57
  • Thanks, Mawg. I've added a better explanation in my original post. I don't think I'm allowed to share the details of what we pay, but it is fairly cheap compared to many of the commercial embedded databases we evaluated. – Fredrik S Oct 22 '18 at 15:15
2

SQLite is ok, but don't plan to use if you plan to insert, update and delete data that involves more that 6 millon rows(All at the same time, or any partial part). The thing is that the VACCUM keyword has to be done everynow and then and it becomes a very severe bottleneck for performance, even when it's automatic.

jRam90
  • 183
  • 1
  • 2
  • 12
2

I will suggest sqlite3 too. It is used by many famous application.

sagasw
  • 133
  • 1
  • 7
1

i am not familiar with embed system, but iphone use arm9, and sqlite as DB

joetsuihk
  • 536
  • 1
  • 3
  • 14
0

The 01-11-10 Embedded.com Newsletter does a nice job of covering this topic. The newsletter can be found at Embedded.com: Embedded.com Tech Focus Newsletter (1-11-10): Embedding Databases.

CPlusPlus OOA and D
  • 420
  • 4
  • 8
tkyle
  • 151
  • 1
  • 3
  • 9