Android SqlChiper Upgrade from 4.2.0 to 4.6.1 for 16kb devices support

coder123 · January 5, 2025, 1:01pm

Hello,
I am upgrading sqlChiper library in my project to support 16kb devicces,
from: implementation 'net.zetetic:android-database-sqlcipher:4.2.0@aar'
to: implementation 'net.zetetic:sqlcipher-android:4.6.1@aar'

during this update some I have noticed that somefunctions were modified,
currently I am getting a crash, when using the SQLiteDatabaseHook
I have noticed that the SQLiteDatabaseHook preKey and postKey were changed and now are receiving connection: SQLiteConnection? in the functions,
previously it was SQLiteDatabase and I used
database.rawExecSQL("PRAGMA cipher_migrate") which worked, now the equivalent
connection?.executeRaw("PRAGMA cipher_migrate", null, null) is causing a crash, but was sufficient up until 4.2.0.

I have also read some previous comments and saw that there are some more queries that might resolve the issue, I have also tried

SQLiteDatabaseHook hook = new SQLiteDatabaseHook() {
    @Override
    public void preKey(SQLiteConnection connection) {

    }

    @Override
    public void postKey(SQLiteConnection connection) {
        if (connection != null) {
            connection.execute("PRAGMA cipher_compatiblity = 3;", null, null);
            connection.execute("PRAGMA cipher_page_size = 4096;", null, null);
            try {
                connection.execute("ATTACH DATABASE 'history_log_new.db' as sqlcipher4 KEY '" + password + "'", null, null);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
            connection.execute("SELECT sqlcipher_export('sqlcipher4')", null, null);
            connection.execute("DETACH DATABASE sqlcipher4;", null, null);
        }
}};

But this is throwing the following:
android.database.sqlite.SQLiteException: file is not a database (code 26): , while compiling: ATTACH DATABASE 'history_log_new.db' as sqlcipher4 KEY ‘…..’

Any guidance would be appreciated,
thanks

developernotes · January 6, 2025, 1:32pm

Hi @coder123,

You should not need to set cipher_compatibility or cipher_page_size unless those values were previously used to configure access to the database in question. In terms of migrating your database, if you were previously able to access your database without invoking cipher_migrate, you would not need to invoke that either as it’s purpose is to migrate SQLCipher database files from previous major versions of SQLCipher (e.g., a SQLCipher 3 database file migrating to SQLCipher 4).

Should you need to perform a cipher_migrate command within the postKey hook, you can use the method executeForLong to check to the result of your migration. An example is available within our test suite here.

coder123 · January 6, 2025, 2:56pm

Hi,
Thanks for the comment!
This solution works on a normal emulator, but when I try it on a SDK 35 16kb version emulator it crashes with the following as soon as I try to write to the db:
Failed to open database ‘/data/user/0/com.app/databases/user’.
android.database.sqlite.SQLiteException: SQL logic error (code 1): , while compiling: SELECT COUNT(*) FROM sqlite_schema;
any advice?

developernotes · January 7, 2025, 6:07pm

Hi @coder123,

I just ran through a small test using SQLCipher for Android 4.6.1 Community edition on both the standard API 35 and API 35 (16kb page size) emulators, both operated properly.

Can you create a small, standalone sample application and publish it on GitHub that reproduces the scenario you are seeing in isolation? We’d be happy to look into this further. Thanks!

coder123 · January 8, 2025, 1:14pm

Hi @developernotes
Thanks a lot for trying to resolve this with me, can I see the project you tried running with the API 35 (16kb page size edition) for compheresent (is there any git repo with it, or can I ask you to push it somewhere so I can have a look at it please?

even if I will create some side project that would work on a 16kb emulator this would not help me since my goal is to make the primary project work, it works great on a normal emulator, but I was unsecseful in resolving the crash on the 16kb version ones, do you maybe have some kind of suggestions or ideas why is this happening? any lead would be very appriciated,
thanks!

developernotes · January 9, 2025, 7:56pm

Hi @coder123,

I used the sample source we make available within our online integration instructions here. If you are able to create a reproducible scenario we would be happy to investigate further.

The benefit in creating an isolated reproduction of the issue is to verify the issue exists outside of your primary project work. If you’re unable to reproduce the issue in isolation, you might compare the differences in your API usage from the demo (that works) in relation to your primary project (which doesn’t) for any differences.

coder123 · January 13, 2025, 5:41pm

Hi @developernotes,I hope you had a good weekend,
after debugging the latest code I came to realize it is crashing in the openOrCreateDatabase function,
this is my hook class:
class MyDatabaseHook() : SQLiteDatabaseHook {
override fun preKey(connection: SQLiteConnection?) {}

override fun postKey(connection: SQLiteConnection?) {
     connection?.executeForLong("PRAGMA cipher_migrate", null, null)
}

}

I use it like this:

    public SQLiteDatabase open() throws Exception {
        MyDatabaseHook hook = new MyDatabaseHook();
        return SQLiteDatabase.openOrCreateDatabase(databaseFile.getAbsolutePath(), getKey(), null, null, hook);
    }

this function is throwing Method threw 'android.database.sqlite.SQLiteException' exception. with detailMessage: SQL logic error (code 1): , while compiling: SELECT COUNT(*) FROM sqlite_schema;

I have also debbugged the code of the sqlCHiper and saw that exception is thrown in the class:
public final class SQLiteConnection
in the function:

public long executeForLong(String sql, Object[] bindArgs,
            CancellationSignal cancellationSignal)

the exception caught in the debug is shown in the attached screen shot,
android.database.sqlite.SQLiteDoneException

after running the PRAGMA cipher_migrate command,

please advice,
thanks!

developernotes · January 14, 2025, 1:13pm

Hi @coder123,

Have you tried to perform a PRAGMA cipher_migrate on the database in question using a SQLCipher 4.x command line tool? If not, you should try that and see if you’re able to access the database afterwards. If you have, what were your results? Finally, having a small reproduction case of your issue in a public repository would be best to allow us to further investigate.

coder123 · January 19, 2025, 11:56am

Hi @developernotes!

I’ll try to explain the issue in detail. The migration seems fine, and everything works correctly when I update from an older version using SQLCipher 3.5.9 to the latest version 4.6.1 (on a non 16kb emulator). The problem arises in a specific scenario:

On a 16 KB device, with a clean install (with or without the hook), everything works as expected.
However, when updating from an older version, the app crashes when trying to create the database using the following code:

SQLiteDatabase.openOrCreateDatabase(databaseFile.getAbsolutePath(), getKey(), null, null, hook);

The crash happens within the SQLCipher code, throwing the following exception:
android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: SELECT COUNT(*) FROM sqlite_schema;

the hook class:

class MyDatabaseHook : SQLiteDatabaseHook {
    override fun preKey(connection: SQLiteConnection?) {}

    override fun postKey(connection: SQLiteConnection?) {
        connection?.executeRaw("PRAGMA cipher_migrate", null, null)
    }
}

I have tried many ways in all possible combinations, also with the executeForLong like you suggested,

this issue does not occur on non-16 KB devices or emulators, where the migration works as expected.
if try a clean install without migration from sqlchiper 3, it also works fine, on both 16kb and non 16kb emulators,

Any guidance or suggestions would be greatly appreciated!

sjlombardo · January 19, 2025, 8:39pm

Hello @coder123 - 3.5.9 would not run on a 16KB emulator. Thus it is not clear how you would even be testing a migration scenario between an earlier version.

Since we have gone back and forth on this thread 8 or 9 times already, and have thus far been unable to reproduce, it would be best if you would create a small standalone android application that demonstrates the exact problem you are experiencing. Then push it to github and we will download it and attempt to troubleshoot.

coder123 · January 20, 2025, 8:04am

Hi @sjlombardo ,

I am installing a 3.5.9 version which is currently in the play store, it crashes of course, and then I try to update to the newest version of the 4.6.1, when ever the 16kb devices will be released the some of the users will have the old version pre-installed, therefore I am trying to address an updating issue I am having…

The exception occurs exclusively when upgrading from version 3 to version 4 on a 16KB device. How can I create a standalone app to replicate this behavior? Were there no other reports highlighting this issue?

Thanks!

sjlombardo · January 20, 2025, 3:53pm

@coder123 - no we have not had any other reports of issues related 16kb devices or emulators.

If you install your 3.5.9 application on a 16k emulator, run it, experience a crash, then install the 4.6.1version, are you able to reproduce the problem?

coder123 · January 21, 2025, 9:11am

@sjlombardo
Yes, this is what I decribed in the last comment, when trying to creat the db

SQLiteDatabase.openOrCreateDatabase(databaseFile.getAbsolutePath(), getKey(), null, null, hook);

The crash happens within the SQLCipher code, throwing the following exception:
android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: SELECT COUNT(*) FROM sqlite_schema;
the hook class:

class MyDatabaseHook : SQLiteDatabaseHook {
    override fun preKey(connection: SQLiteConnection?) {}

    override fun postKey(connection: SQLiteConnection?) {
        connection?.executeRaw("PRAGMA cipher_migrate", null, null)
    }
}

sjlombardo · January 21, 2025, 2:34pm

@coder123 great, then what I would recommend doing this:

Create a new standalone android application project that establishes a database using 3.5.9 and commit it to git.
Run that on the 16kb emulator
Create a new branch in the git repo, and upgrade the project to 4.6.1, and implement the migration in the same way you do in the same way you are doing in your project
Deploy that to the android 16kb emulator, and see if you are able to reproduce the crash you described
If you are able to reproduce, publish the git repo to github (including both the original and migration branches). We will then download it and troubleshoot it with the minimal reproduction.
If you are not able to reproduce, compare the new implementation to your application to determine what is different.

Topic		Replies	Views
Upgrading Sqlcipher for Android from 3 to 4 Issues	6	4603	October 7, 2020
Upgrading from android-database-sqlcipher to sqlcipher-android SQLCipher	45	72	December 6, 2024
Breaking changes migrating from android-database-sqlcipher 4.5.3 to sqlcipher-android 4.6.1 Issues	5	29	December 19, 2024
Migrating Android Application from SQLCipher 3 to 4	5	2257	October 1, 2019
How to properly use SQLCipher 4.5.0 on Android? SQLCipher	2	1113	February 4, 2022

Android SqlChiper Upgrade from 4.2.0 to 4.6.1 for 16kb devices support

Related topics