4

I am writing an application on iOS that uses sqlite3 wrapped by fmdatabase. I am running into the problem that at some point, my program is stuck in a loop inside of FMDatabase libraries, particularly, a function that calls sqlite3_step and finds out that the database is busy, then retries over and over again.

I am looking for general debugging tools and tips, as it will be too much to give my whole setup here. Some things that might be of consequence, I opened a database handle to a database that already has a handle in another thread. sqlite3_threadsafe() returns 2, so I know its enabled. I have also tested this new connection by making a very simple select and update statement. When I let my program run, and when it tries to run an update on the database, i get stuck.

The update statement that my program makes itself is not at fault, because this query runs fine when I do NOT open up two connections. Yet, I fail to see where I could be going wrong...

Any help or tips on where I might be wrong would be greatly appreciated.

Ying
  • 1,944
  • 5
  • 24
  • 38
  • did you commit the update? Also see [the unlock notify scheme](http://www.sqlite.org/unlock_notify.html) – Benoit Apr 20 '11 at 15:32
  • fmdatabase calls sqlite3_finalize but not commit. i committed it, but did not seem to make any difference – Ying Apr 20 '11 at 16:35

1 Answers1

21

SQLite locks the entire database during a write operation (i.e. when a write is happening on any table, no other write, to any table anywhere can happen at the same time). Some databases provide concurrent writes via table-level locks, or sometimes row-level locks. To contrast this to SQLite's implementation, a table-level lock basically means that when you're writing data to a given table, no other thread can write to any record in that table, at the same time (however, writes to other tables can occur simultaneously, in some circumstances). Similarly, row-level locks take it even further, and allow only the necessary rows involved to be locked, allowing concurrent writes to the same table to happen from multiple threads. The idea here is to minimize the amount of data you need to lock for a write operation, which effectively increases the amount of concurrent writes possible across the database, and depending on your implementation/how you use your database, this can significantly increase throughput.

Now, back to your question...

The fact that SQLite is threadsafe doesn't mean that multiple threads can concurrently write to it - it means that it has a way of handling access from multiple threads - which is to (a) allow timeouts/retries, and (b) to return a useful error (SQLITE:Busy) when a lock is currently held on the database. That is, threadsafe means nothing more than, "Multiple threads can access this data in a way that won't result in data corruption due to simultaneous access."

Basically, somewhere in the code, one thread is trying to do its update before another thread has released its lock on the database. This is a common hurdle with SQLite, because the authors/documentation will tell you that SQLite can handle concurrency like a champ. The reality is that what SQLite considers "concurrency support" amounts to trying to be very fast so that locks on the database are only held for a very short time, and therefore locks on the database are released before timeouts are hit. In a lot of cases, this works just fine and never gets in your way. However, having very short-lived locks is not the same as actually allowing concurrent writes from multiple threads.

Think of it like the way that iOS does multitasking (at least as of iOS 5, when I'm writing this) - really what it's doing is putting other apps on pause, and coming back to them. This has the effect that (a) battery life is much better due to lower CPU utilization, and (b) you don't have to start an app from scratch every time you launch it. This is great, but the actual word "multitasking" as used in iOS doesn't technically mean the same thing as "multitasking" in other environments (even Mac OS X).

SQLite is the same way. Do they have "concurrency" support? Well sort of, but the way they define the word "concurrency" isn't the way the rest of the DB world defines "concurrency".

No one is really wrong, but in cases like these, it adds to implementation confusion.

jefflunt
  • 33,527
  • 7
  • 88
  • 126
  • yea, i had to protect the database with a lock of my own, which i have tried to avoid but could not... – Ying Apr 27 '11 at 15:33
  • 2
    Now they have something called WAL. – Alex Jan 17 '13 at 21:17
  • 2
    Write-ahead logging will help diminish the symptoms, but still does not handle the case of multiple, concurrent writes. – jefflunt Oct 23 '13 at 14:21
  • 1
    This is the most succinct and useful overview of multi-threading with SQLite that I can find anywhere. Extremely helpful, thank you. – Richard Dec 13 '15 at 16:35
  • @Richard - Thanks. I've been out of practice using SQLite for many years now, so I hope it's still an answer that is accurate to SQLite's implementation. – jefflunt Dec 13 '15 at 20:12