1

I'm making a stress test for in-memory SQLite db in iOS.

After for a while, OCUnit fired this error.

otest(79450,0xad21a2c0) malloc: *** mmap(size=40267776) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

I think this is hitting memory limit. But I can't figure out my current database size. I tried,

sqlite3_status(SQLITE_STATUS_MEMORY_USED, &current, &peak, false);

but this always returns 0 for current, and some value for peak all same over updates. I read manual entry, but it doesn't look like db size feature. I think I'm digging wrong place. How can I get database size with C-API?

eonil
  • 83,476
  • 81
  • 317
  • 516
  • I assume current/peak are int. why not test the return code to make sure it's SQLITE_OK. Have you sprinkled these calls around in your code so you know for sure 0s always returned? Do check the return code and update question. – David H Jul 22 '12 at 12:55
  • Ah actually I meant `peak` is returning same value over updated regardless of how many data inserted. I'm sorry for the ambiguity. And of course, I'm doing error checking for the call. – eonil Jul 22 '12 at 14:20
  • If it takes a while for this error to happen, are you sure you don't have any sqlite3 related leak? E.g. is every prepare statement partnered with a finalize statement? – Rob Jul 22 '12 at 15:13
  • @RobertRyan Actually, I'm writing this stress test to discover those kind of leaks. The issue is I just want to know how big the in-memory database is... – eonil Jul 22 '12 at 20:07

1 Answers1

0

Based on your comments and the mmap error, I think you are having a similar problem to one I am still struggling with - over use of unified buffer cache. What I'm guessing is going on here is that sqlite is using a mmap file to backup its "in memory" database. So its mmap'd memory, which uses the unified buffer cache (i.e. the shared memory pool), and probably does not use much malloc'd memory at all (which is why you don't see big numbers there).

Now, as the ubc cache becomes increasingly "dirty", iOS starts getting deprived of memory it can use at any instance. So, at some time, sqlite tries to mmap more memory, but when the system looks for free ubc pages, there aren't any free ones. It takes a LONG time to flush blocks to a flash file system.

What makes this such an onerous problem is there is little you can do in your app to even discover that memory is getting low, so you can take steps to avoid it.

What I did in my "display huge downloaded images" open source project is sep track of my ubc usage, and never let it get above 50% of the total system memory.

In your case you may find it better to have sqlite to be file based, then save and F_FULLSYNC the file (waiting for completion) before proceeding. In the end I funneled all my image work through a serial dispatch queue that keep account of the high water mark.

David H
  • 40,852
  • 12
  • 92
  • 138
  • thanks for interesting informations :) It was just a test, and I wanted to know why this happens. Anyway the manual emphasized the in-memory db (named `:memory:`) will be remain in memory purely (http://www.sqlite.org/inmemorydb.html). Should I accept this situation as SQLite on iOS behaves differently with manual? – eonil Jul 22 '12 at 17:39
  • Look, I don't know sqlite internals. In memory can certainly mean mmap'd memory, means the memory is mapped into at temp file so it does not all have to be resident at the same time. Or, it just may mean that sqllite tried to map 40M of memory (additional) and it could not get that much memory from the system. I spent a week tracking down my issue so it might well take you some time and digging too. What caught my attention was that sqllite was trying to mmap 40M of memory but you said your current or peak memory was no where near that much. – David H Jul 22 '12 at 18:21