net.sqlcipher.database.SQLiteException: file is not a database: , while compiling: select count(*) from sqlite_master;

[pallavi]

Hi,

Our application was using older version of SqlCipher i.e. sqlcipher.jar we create the backup of data using older version of sqlcipher.

But when we try to open the backup db file using new version net.zetetic:android-database-sqlcipher:4.0.1@aar its giving following errors:

net.sqlcipher.database.SQLiteException: file is not a database: , while compiling: select count(*) from sqlite_master;
    at net.sqlcipher.database.SQLiteCompiledSql.native_compile(Native Method)
    at net.sqlcipher.database.SQLiteCompiledSql.compile(SQLiteCompiledSql.java:91)
    at net.sqlcipher.database.SQLiteCompiledSql.<init>(SQLiteCompiledSql.java:64)
    at net.sqlcipher.database.SQLiteProgram.<init>(SQLiteProgram.java:89)
    at net.sqlcipher.database.SQLiteQuery.<init>(SQLiteQuery.java:48)
    at net.sqlcipher.database.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:60)
    at net.sqlcipher.database.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1954)
    at net.sqlcipher.database.SQLiteDatabase.rawQuery(SQLiteDatabase.java:1840)
    at net.sqlcipher.database.SQLiteDatabase.keyDatabase(SQLiteDatabase.java:2571)
    at net.sqlcipher.database.SQLiteDatabase.openDatabaseInternal(SQLiteDatabase.java:2502)
    at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:1204)
    at net.sqlcipher.database.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:1267)
    at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:162)
    at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:131)

[sjlombardo]

see Upgrading to SQLCipher 4

[pallavi]

But our old application is eclipse based and we are using sqlcipher.jar how to upgrade that ??

[sjlombardo]

@pallavi - Since you were previously using the old style integration where SQLCipher for Android was distributed in a .zip package and files were manually copied to the application project, you’ll need to remove the old builds first. Start by removing each of the platform specific libraries (i.e. libdatabase_sqlcipher.so , libsqlcipher_android.so , and libstlport_shared.so ). Then remove the sqlcipher.jar and finally the ICU zip file, icudt46l.zip, which was previously included within the assets directory of your application. Once all the old references are removed you can proceed with the AAR packages integration as described here:

https://www.zetetic.net/sqlcipher/sqlcipher-for-android/

[pallavi]

@sjlombardo - Our Eclipse based application is already released in the market and now we have migrated Our Application to Android Studio.
Our use case is if user is taking backup of existing data which we encrypted using old sqlcipher.jar then we need to decrypt it using new version net.zetetic:android-database-sqlcipher:4.0.1@aar Which is giving crash.
Please provide suggestion how to give backward compatibility in Android studio with Sqlcipher new version.

[sjlombardo]

Hello @pallavi - If you have preexisting databases that were created with prior versions of SQLCipher you would need to either migrate the databases or run in compatibility mode. The detailed options are described here:

Upgrading to SQLCipher 4

The recent release of SQLCipher 4 introduces many new performance and security enhancements for applications that use SQLCipher for secure local data storage. However, the introduction of new default algorithms, increased KDF iterations and a larger page size means that SQLCipher 4 will not open older databases by default. This document provides guidance on the upgrade options available to applications that have previously integrated SQLCipher versions 1 through 3. Option 1: Database File Migr…

[pallavi]

Hi Stephen,

I was trying to implement Option 1 mentioned in your Answer that is Option 1: Database File Migration
But i did not understand where to add PRAGMA cipher_migrate and SQLITE_OPEN_CREATE flag
i am using getWritableDatabase(password) to open the database file.

[pallavi]

Even to run in compatibility mode where to Add below code ???

PRAGMA cipher_page_size = 1024;
PRAGMA kdf_iter = 4000;
PRAGMA cipher_hmac_algorithm = HMAC_SHA1;
PRAGMA cipher_kdf_algorithm = PBKDF2_HMAC_SHA1;
PRAGMA cipher_use_hmac = OFF;

[sjlombardo]

@pallavi - Here is an example of using cipher_migrate via postKey.

sqlcipher/sqlcipher-android-tests/blob/master/app/src/main/java/net/zetetic/tests/CipherMigrateTest.java

You can also do same with cipher_compatibility.

[developernotes]

Hi @pallavi

Please call SQLiteDatabase.loadLibs before your constructor call to SecureAppsDataProvider. Also, it is recommended to validate to result code of the pragma cipher_migrate command as provided in the example.

[pallavi]

Hi @developernotes

1. If i pass hook object in SQLiteOpenHelper constructor then for every call i have to execute
   below command every time.
   What impact it will have on performance??

SQLiteDatabaseHook hook = new SQLiteDatabaseHook() {
public void preKey(SQLiteDatabase database) {}
public void postKey(SQLiteDatabase database) {
database.rawExecSQL(“PRAGMA cipher_migrate”);
}
};

2. Is there any way to read meta data from .db files to get to know that it was encrypted using old SqlCipher Version???

[sjlombardo]

Hello @pallavi. It is not possible to determine from the database file what version was used to encrypt a database. You can certainly keep this metadata yourself, either in application preferences or even encoring it as part of the file name. Alternatively, you can use the following standard process for opening and upgrading databases.

1. Attempt to open the database using standard settings (i.e. whatever the defaults are for the current version of SQLCipher for the app (in this case SQLCipher 4).
2. If the database can’t be opened using the key and the default settings, try to open it and run PRAGMA cipher_migrate on it (e.g. with postKey in the case of android). This would then attempt to upgrade the database. If the migration succeeds, you can continue to use that connection for the remainder of the application lifecycle. If the key is incorrect here, then migration will not occur and the database will remain untouched.
3. If step 1 and step 2 fail, then the key material is incorrect or the settings of the database were not consistent with defaults for previous SQLCipher verions (i.e. custom settings were used that require manual migration)

This approach has the benefit of performing optimally in the standard case when the database has already been migrated. It has a slowdown in the event that the key material is incorrect because the key may be derived multiple times to attempt migration, but that usally acceptable in most cases.

In the event that incorrect keys are a common situation, and thus the performance hit for rechecking in step 2 is not acceptable, then the other approaches you mentioned are more suitable, i.e. statefully tracking the current version of the database in an application preference.

[pallavi]

Hi @sjlombardo

What is PreKey and PostKey and when it gets called ??

[developernotes]

Hi @pallavi

The preKey and postKey events are invoked around the keying operation of the database. You can see the call sites here.

[pallavi]

hi @developernotes I encrypt my database using net.zetetic:android-database-sqlcipher:4.0.1@aar in one device and try to decrypt it in another device with older Sqlcipher.jar
with the same password but it says file is not a database and application crashes. Could you please suggest how to handle this ??

[developernotes]

Hi @pallavi

@pallavi

try to decrypt it in another device with older Sqlcipher.jar

What version of SQLCipher is the jar file? If you can execute code using it, try running:

PRAGMA cipher_version;

You might consider updating to the latest version of SQLCipher. If the version is a different major version, you will want to review the upgrade guidance here:

Upgrading to SQLCipher 4

The recent release of SQLCipher 4 introduces many new performance and security enhancements for applications that use SQLCipher for secure local data storage. However, the introduction of new default algorithms, increased KDF iterations and a larger page size means that SQLCipher 4 will not open older databases by default. This document provides guidance on the upgrade options available to applications that have previously integrated SQLCipher versions 1 through 3. Option 1: Database File Migr…

[pallavi]

Hi @developernotes

jar version is : Implementation-Version: v3.2.0
our application is already released in the market for with older device.

there’s a use case if user want to open db files of new device into old device.
it should support that operation.

but since in new device files are encrypted using this net.zetetic:android-database-sqlcipher:4.0.1@aar and decryption will happen sqlcipher. jar version v3.2.0 it crashes.

since we can not make any changes to the older released Application this problem is coming please suggest something.

[sjlombardo]

Hello @pallavi, since SQLCipher 4 uses new settings and algorithms that are not available in SQLCipher 3 there is really only one option. That would be to have you new application version use SQLCipher 4 but create / manage databases in version 3 format, e.g. using PRAGMA cipher_compatibility. This would enable you to create databases that can be opened by SQLCipher 3.

[pallavi]

hi @sjlombardo i am executing below code

SQLiteDatabaseHook hook = new SQLiteDatabaseHook() {
public void preKey(SQLiteDatabase database) {
}
public void postKey(SQLiteDatabase database) {
database.rawExecSQL(“PRAGMA cipher_compatibility=3;”);
database.rawExecSQL(“PRAGMA kdf_iter=1000;”);
database.rawExecSQL(“PRAGMA cipher_default_kdf_iter=1000;”);
database.rawExecSQL(“PRAGMA cipher_page_size = 4096;”);
}
};

but it crashes and says net.sqlcipher.database.SQLiteException: file is encrypted or is not a database