net.sqlcipher.database.SQLiteException: file is not a database: , while compiling: select count(*) from sqlite_master;

Hi,

Our application was using older version of SqlCipher i.e. sqlcipher.jar we create the backup of data using older version of sqlcipher.

But when we try to open the backup db file using new version net.zetetic:android-database-sqlcipher:4.0.1@aar its giving following errors:

net.sqlcipher.database.SQLiteException: file is not a database: , while compiling: select count(*) from sqlite_master;
    at net.sqlcipher.database.SQLiteCompiledSql.native_compile(Native Method)
    at net.sqlcipher.database.SQLiteCompiledSql.compile(SQLiteCompiledSql.java:91)
    at net.sqlcipher.database.SQLiteCompiledSql.<init>(SQLiteCompiledSql.java:64)
    at net.sqlcipher.database.SQLiteProgram.<init>(SQLiteProgram.java:89)
    at net.sqlcipher.database.SQLiteQuery.<init>(SQLiteQuery.java:48)
    at net.sqlcipher.database.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:60)
    at net.sqlcipher.database.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1954)
    at net.sqlcipher.database.SQLiteDatabase.rawQuery(SQLiteDatabase.java:1840)
    at net.sqlcipher.database.SQLiteDatabase.keyDatabase(SQLiteDatabase.java:2571)
    at net.sqlcipher.database.SQLiteDatabase.openDatabaseInternal(SQLiteDatabase.java:2502)
    at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:1204)
    at net.sqlcipher.database.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:1267)
    at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:162)
    at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:131)

see Upgrading to SQLCipher 4

But our old application is eclipse based and we are using sqlcipher.jar how to upgrade that ??

@pallavi - Since you were previously using the old style integration where SQLCipher for Android was distributed in a .zip package and files were manually copied to the application project, you’ll need to remove the old builds first. Start by removing each of the platform specific libraries (i.e. libdatabase_sqlcipher.so , libsqlcipher_android.so , and libstlport_shared.so ). Then remove the sqlcipher.jar and finally the ICU zip file, icudt46l.zip, which was previously included within the assets directory of your application. Once all the old references are removed you can proceed with the AAR packages integration as described here:

https://www.zetetic.net/sqlcipher/sqlcipher-for-android/

@sjlombardo - Our Eclipse based application is already released in the market and now we have migrated Our Application to Android Studio.
Our use case is if user is taking backup of existing data which we encrypted using old sqlcipher.jar then we need to decrypt it using new version net.zetetic:android-database-sqlcipher:4.0.1@aar Which is giving crash.
Please provide suggestion how to give backward compatibility in Android studio with Sqlcipher new version.

Hello @pallavi - If you have preexisting databases that were created with prior versions of SQLCipher you would need to either migrate the databases or run in compatibility mode. The detailed options are described here:

Hi Stephen,

I was trying to implement Option 1 mentioned in your Answer that is Option 1: Database File Migration
But i did not understand where to add PRAGMA cipher_migrate and SQLITE_OPEN_CREATE flag
i am using getWritableDatabase(password) to open the database file.

Even to run in compatibility mode where to Add below code ???

PRAGMA cipher_page_size = 1024;
PRAGMA kdf_iter = 4000;
PRAGMA cipher_hmac_algorithm = HMAC_SHA1;
PRAGMA cipher_kdf_algorithm = PBKDF2_HMAC_SHA1;
PRAGMA cipher_use_hmac = OFF;

@pallavi - Here is an example of using cipher_migrate via postKey.

You can also do same with cipher_compatibility.

Hi @pallavi

Please call SQLiteDatabase.loadLibs before your constructor call to SecureAppsDataProvider. Also, it is recommended to validate to result code of the pragma cipher_migrate command as provided in the example.

Hi @developernotes

  1. If i pass hook object in SQLiteOpenHelper constructor then for every call i have to execute
    below command every time.
    What impact it will have on performance??

SQLiteDatabaseHook hook = new SQLiteDatabaseHook() {
public void preKey(SQLiteDatabase database) {}
public void postKey(SQLiteDatabase database) {
database.rawExecSQL(“PRAGMA cipher_migrate”);
}
};

  1. Is there any way to read meta data from .db files to get to know that it was encrypted using old SqlCipher Version???

Hello @pallavi. It is not possible to determine from the database file what version was used to encrypt a database. You can certainly keep this metadata yourself, either in application preferences or even encoring it as part of the file name. Alternatively, you can use the following standard process for opening and upgrading databases.

  1. Attempt to open the database using standard settings (i.e. whatever the defaults are for the current version of SQLCipher for the app (in this case SQLCipher 4).
  2. If the database can’t be opened using the key and the default settings, try to open it and run PRAGMA cipher_migrate on it (e.g. with postKey in the case of android). This would then attempt to upgrade the database. If the migration succeeds, you can continue to use that connection for the remainder of the application lifecycle. If the key is incorrect here, then migration will not occur and the database will remain untouched.
  3. If step 1 and step 2 fail, then the key material is incorrect or the settings of the database were not consistent with defaults for previous SQLCipher verions (i.e. custom settings were used that require manual migration)

This approach has the benefit of performing optimally in the standard case when the database has already been migrated. It has a slowdown in the event that the key material is incorrect because the key may be derived multiple times to attempt migration, but that usally acceptable in most cases.

In the event that incorrect keys are a common situation, and thus the performance hit for rechecking in step 2 is not acceptable, then the other approaches you mentioned are more suitable, i.e. statefully tracking the current version of the database in an application preference.

Hi @sjlombardo

What is PreKey and PostKey and when it gets called ??

Hi @pallavi

The preKey and postKey events are invoked around the keying operation of the database. You can see the call sites here.