Intermittent NPE when inserting data into SQLite

Question

I'm getting an NullPointerException when I insert values into to my SQLite table on Android and I don't understand why. I'm testing ContentValues and the database instance for null.

This is the insertion code:

public void insertOrIgnore(ContentValues values) {

    SQLiteDatabase db = this.dbHelper.getWritableDatabase();

    try {

        //I added these null value checks to stop NPE, but doesn't help.
        if (values != null && db != null) {
            db.insertWithOnConflict(TABLE, null, values, SQLiteDatabase.CONFLICT_IGNORE);
        }
    } catch (SQLiteException e) {

    } finally {
        if (db != null) {
            db.close();
        }
    }
}

where

public static final String TABLE = "albums";

Most of the time this code works with the data added to the database as expected. However, it sometimes and rarely generates the below error. The stack trace is from ACRA and I have not been able to isolate under what conditions this error occurs. I'm looking for pointers as to why this happens and what the conditions are. My knowledge of SQLite is beginner level.

java.lang.NullPointerException
    at android.database.sqlite.SQLiteStatement.releaseAndUnlock(SQLiteStatement.java:290)
    at android.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:96)
    at android.database.sqlite.SQLiteDatabase.executeSql(SQLiteDatabase.java:2025)
    at android.database.sqlite.SQLiteDatabase.execSQL(SQLiteDatabase.java:1965)
    at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:690)
    at android.database.sqlite.SQLiteDatabase.beginTransactionNonExclusive(SQLiteDatabase.java:605)
    at android.database.sqlite.SQLiteStatement.acquireAndLock(SQLiteStatement.java:247)
    at android.database.sqlite.SQLiteStatement.executeInsert(SQLiteStatement.java:112)
    at android.database.sqlite.SQLiteDatabase.insertWithOnConflict(SQLiteDatabase.java:1844)
    at com.mydomain.myapp.albums.AlbumsData.insertOrIgnore(AlbumsData.java:89)

Line 89 is the db.insertWithOnConflict(...) call shown above.

I'm not looking for an answer with complete code necessarily but rather a pointer and explanation as to what's going wrong so I can begin to fix it myself.

EDIT: The stack trace shows the NPE originates from line 290 of SQLiteStatement (v 4.03):

 setNativeHandle(mDatabase.mNativeHandle);

So it seems the database instance is null. How can it become null during a transaction when I tested for null at the beginning of the transaction?

I've updated the question with TABLE's value. It's a static field in the AlbumsData class where insertOrIgnore is a method. It's initialized as shown above. — Pierre Rymiortz, Nov 26 '12 at 13:22
Line 89 is db.insertWithOnConflict(...). I've edited the question to clarify this. — Pierre Rymiortz, Nov 26 '12 at 13:22
http://stackoverflow.com/q/7999075/995891 should be the same problem. Looks like a framework bug to me. — zapl, Nov 26 '12 at 13:43
@zapl Your link is very valuable in helping me understand the issue. I've removed the db.close statement as suggested in the link and so far in testing it hasn't crashed again. Please put it in an answer and I'll accept it as the answer. — Pierre Rymiortz, Nov 26 '12 at 15:54

score 2 · Accepted Answer · edited May 23 '17 at 12:12

As mentioned here SQLiteDatabase close() function causing NullPointerException when multiple threads

The reason for your bug could be that you close the database at some point. Probably concurrently while the task that fails was not finished.

I've followed the stacktrace a bit and this is what roughly happens:

AlbumsData.insertOrIgnore(AlbumsData.java:89)
You call insertWithOnConflict, which builds the resulting sql string ("INSERT OR IGNORE INTO...") then wraps that together with the values from your ContentValues into a SQLiteStatement.
SQLiteDatabase.insertWithOnConflict(SQLiteDatabase.java:1844) - The resulting statement is to be executed now
SQLiteStatement.executeInsert(SQLiteStatement.java:112) - before the actual insert can happen, the database needs to acquire a lock.
SQLiteStatement.acquireAndLock(SQLiteStatement.java:247) - some checks happen here, the database object is as far as I can see not null at that point. Code decides that it has to start a transaction. The database object itself is as far as I can see not locked at that point.
SQLiteDatabase.beginTransactionNonExclusive(SQLiteDatabase.java:605) - just forwarding
SQLiteDatabase.beginTransaction(SQLiteDatabase.java:690) - after some checks (not sure if database has to exist here) it will try to execute execSQL("BEGIN IMMEDIATE;")
SQLiteDatabase.execSQL(SQLiteDatabase.java:1965) - just forward
SQLiteDatabase.executeSql(SQLiteDatabase.java:2025) - builds another SQLiteStatement out of "BEGIN IMMEDIATE;. This one should be executed now
SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:96) - starts with checking the database lock, this seems to be okay and the database should not be null here. The statement is then executed and finally the database is to be unlocked again.
SQLiteStatement.releaseAndUnlock(SQLiteStatement.java:290) - cleans up some stuff and in the end fails with NPE because the database is null.

Line numbers don't match so there are probably vendor modifications / additions in that code.

As you can see, the code crashes before actually using the data you supplied. It was about to do

BEGIN TRANSACTION IMMEDIATE; -- crash
INSERT INTO table (...) VALUES (...);
-- (end transaction)

That makes it in my opinion a framework bug. The database object that is internally handled there should not be able to be null somewhere down the line, especially when it seems that it was not null further up in the stack.

I also think that it is possible that another hidden exception could be the root cause for this. There are a lot of try { /* do stuff */ } finally { /* clean up */ } blocks within the code and the finally part will be executed even if the try part throws an exception. Now the finally block could cause another exception and the result is AFAIK that the original exception is replaced by the new exception from the finally block.

Especially executeUpdateDelete() is like

try {
    acquireAndLock(WRITE);
    // actual statement execution
} finally {
    releaseAndUnlock();
}

if the database is closed at that point, acquireAndLock or any code in the try part could fail and that could leave the database object at null which causes releaseAndUnlock to fail again. You should get the same stacktrace.

Apart from that, don't do empty catch blocks like catch (SQLiteException e) { /* empty */ }. Log them with ACRA if possible / you don't do that already.

Thank you for such a detailed and thorough answer. It not only summarizes the link but provides new valuable perspective. Thanks also for the advice about empty catch blocks. — Pierre Rymiortz, Nov 26 '12 at 23:05

Mohamed_AbdAllah · Answer 2 · 2012-11-26T14:00:28.920

1

This NPE appears to be from a custom ROM as the Android source code is pointing to different Methods than the ones you receive in the LogCat. What I do for such cases is that: if the rate of these exceptions is very rare, I ignore them as it is difficult to know what custom ROM is running on the phone and more difficult to get the source code of this custom ROM to know where the problem is.

Not many users are using custom ROMs, so if you extensively tested your App on different phones with different SDKs and the rate of the Exceptions you get is not that significant, you can ignore them. Otherwise, you can take a shoot in the dark and speculate what can be in this custom ROM that is causing NPE (personally, I think it is not worth the effort).

edited Nov 26 '12 at 14:00

answered Nov 26 '12 at 13:26

Mohamed_AbdAllah

5,311
3
28
47

Thanks for the answer. The error is from my own phone, occurring while I use the app to download album data from a web site. My phone is running Android 4.03. As far as I can tell the stack trace can be followed through the Android source - see the edit to my question. – Pierre Rymiortz Nov 26 '12 at 14:11
For the other class (android.database.sqlite.SQLiteDatabase) it is not pointing to the methods in the Logcat (for e.g. line `SQLiteDatabase.java:1965` is pointing to `public boolean isReadOnly() {` while in your LogCat it is pointing to `execSQL` (same for the others). Is your phone on a custom ROM? – Mohamed_AbdAllah Nov 26 '12 at 14:36
It's not a custom ROM but you are definitely right about the line numbers for SQLiteDataBase. I've checked all Android source versions for the class and none of the source code line numbers match mine. How curious. +1 to you for noticing it. Any perspective on how this line number puzzle can occur is appreciated (or perhaps it warrants its own SO question...) – Pierre Rymiortz Nov 26 '12 at 14:57
Check the answer below, it is very detailed. What I recommend though to avoid the hassle of openning and closing DBs is to pass the db instance as a parameter to your insertOrIgnore() method and close it outside it (open and close it in the calling method) – Mohamed_AbdAllah Nov 26 '12 at 17:07

Intermittent NPE when inserting data into SQLite

2 Answers2