# Android API Update

**URL:** <https://discuss.zetetic.net/t/android-api-update/3551>\
**Category:** SQLCipher\
**Created:** [March 27, 2019, 11:53pm UTC](https://discuss.zetetic.net/t/android-api-update/3551 "2019-03-27T23:53:22Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeff\_Lockhart](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/jeff_lockhart/32/993_2.png) [@Jeff\_Lockhart](https://discuss.zetetic.net/u/Jeff_Lockhart)\
**Post date:** [March 27, 2019, 11:53pm UTC](https://discuss.zetetic.net/t/android-api-update/3551/1 "2019-03-27T23:53:22Z")

</div>

There have been [several](https://discuss.zetetic.net/t/android-missing-api-method/192) [discussions](https://groups.google.com/forum/#!topic/sqlcipher/4pE_XAE14TY) in years past about updating the SQLCipher Android API to be more in line with the latest platform SQLite implementation. There have been some updates in the recent releases to add new API methods, which have been appreciated, but the code in [`SQLiteDatabase`](https://github.com/sqlcipher/android-database-sqlcipher/blob/a5b9430e88f836cf0981a694d910d8781fe02614/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteDatabase.java) remains largely unchanged from the [Android Gingerbread implementation](https://android.googlesource.com/platform/frameworks/base.git/+/refs/heads/gingerbread-release/core/java/android/database/sqlite/SQLiteDatabase.java). Is it on your road map to revamp this API, bringing the performance improvements of the [latest platform `SQLiteDatabase`](https://android.googlesource.com/platform/frameworks/base.git/+/refs/heads/master/core/java/android/database/sqlite/SQLiteDatabase.java) to SQLCipher? Considering most of these changes were introduced in [Jelly Bean API 16](https://android.googlesource.com/platform/frameworks/base.git/+/refs/heads/jb-release/core/java/android/database/sqlite/SQLiteDatabase.java) and [99.5% of Android devices](https://developer.android.com/about/dashboards) are running Jelly Bean or newer, it would be a much welcomed update.

Specifically I’m interested in improved database concurrency, as this has been a consistent problem in our app using SQLCipher. The current implementation locks the database on nearly all [accesses](https://github.com/sqlcipher/android-database-sqlcipher/blob/a5b9430e88f836cf0981a694d910d8781fe02614/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteDatabase.java#L1697), [transactions](https://github.com/sqlcipher/android-database-sqlcipher/blob/a5b9430e88f836cf0981a694d910d8781fe02614/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteDatabase.java#L2914), [writes](https://github.com/sqlcipher/android-database-sqlcipher/blob/a5b9430e88f836cf0981a694d910d8781fe02614/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteDatabase.java#L2353), and reads ([before](https://github.com/sqlcipher/android-database-sqlcipher/blob/a5b9430e88f836cf0981a694d910d8781fe02614/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteDatabase.java#L490) and [during](https://github.com/sqlcipher/android-database-sqlcipher/blob/a5b9430e88f836cf0981a694d910d8781fe02614/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteQuery.java#L72)), which prevents [SQLite WAL journal mode](https://www.sqlite.org/wal.html) from allowing [multiple readers along with a single writer](https://www.sqlite.org/wal.html#advantages). Following [the locking pattern](https://android.googlesource.com/platform/frameworks/base.git/+/refs/heads/pie-release/core/java/android/database/sqlite/SQLiteDatabase.java#110) in the latest platform implementation would be a massive improvement in concurrency performance.

---

<div class="post-metadata">

**Author:** ![developernotes](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/developernotes/32/1309_2.png) [@developernotes](https://discuss.zetetic.net/u/developernotes)\
**Post date:** [March 29, 2019, 2:58pm UTC](https://discuss.zetetic.net/t/android-api-update/3551/2 "2019-03-29T14:58:56Z")

</div>

Hello @Jeff_Lockhart

Thanks for reaching out with regard to the SQLCipher for Android API. We certainly agree that making adjustments to the the locking behavior within the library would be a beneficial approach for all users. While we understand the demographic continues to advance, we have long supported a much wider swath of Android API versions which is a consideration when changes are evaluated and introduce to the library. That said, beyond the concurrency issue, are there specific features that you are looking for not present in the API? This is certainly on our radar, however we do not have a concrete timeline yet.

---

<div class="post-metadata">

**Author:** ![Jeff\_Lockhart](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/jeff_lockhart/32/993_2.png) [@Jeff\_Lockhart](https://discuss.zetetic.net/u/Jeff_Lockhart)\
**Post date:** [March 29, 2019, 8:09pm UTC](https://discuss.zetetic.net/t/android-api-update/3551/3 "2019-03-29T20:09:01Z")

</div>

Concurrency performance is the #1 issue we experience using SQLCipher for Android. We see many ANR reports, especially from older devices with slower I/O, that stem from what should be small database reads that occur on the main UI thread, that are blocked on `SQLiteDatabase.mLock` because a background thread is performing another longer running database operation. SQLite, especially in WAL journal mode, should be able to perform these database operations concurrently, but the `SQLiteDatabase` API has this internal lock, which Google’s SQLite API has improved the behavior of since the Jelly Bean platform update.

It’s admirable you’ve maintained backwards compatibility support for so many Android APIs. But even the 0.5% of devices that are running an older platform than Jelly Bean, could still run an app using SQLCipher with an updated `SQLiteDatabase` implementation based on the latest Android platform code. Updating the API would only affect apps that might still be calling one of a few deprecated methods in `SQLiteDatabase`. But there’s really no reason an app would need to be calling those methods, 7 years after they were deprecated. Every one of the deprecated methods either has a safer replacement, serves no purpose anymore, or specifically is used to deal with the poor locking behavior of the older API.

> `yieldIfContended()`
> 
> _This method was deprecated in API level 15. if the db is locked more than once (because of nested transactions) then the lock will not be yielded. Use yieldIfContendedSafely instead._
> 
> `getSyncedTables()`
> 
> _This method was deprecated in API level 15. This method no longer serves any useful purpose and has been deprecated._
> 
> `markTableSyncable(String table, String deletedTable)`
> 
> _This method was deprecated in API level 15. This method no longer serves any useful purpose and has been deprecated._
> 
> `markTableSyncable(String table, String foreignKey, String updateTable)`
> 
> _This method was deprecated in API level 15. This method no longer serves any useful purpose and has been deprecated._
> 
> `setLockingEnabled(boolean lockingEnabled)`
> 
> _This method was deprecated in API level 16. This method now does nothing. Do not use._
> 
> `isDbLockedByOtherThreads()`
> 
> _This method was deprecated in API level 16. Always returns false. Do not use this method._
> 
> There is no longer the concept of a database lock, so this method always returns false.

With the Android platform SQLite code updates, the deprecated methods are still present, so updating the code doesn’t actually break old apps. The methods simply don’t do anything, as they no longer need to. It would work the same updating SQLCipher’s Android API and would bring much improved concurrency performance for all SQLCipher Android apps.

If I had the time to dedicate to it, I would be willing to assist in these updates, certainly with testing.

---

<div class="post-metadata">

**Author:** ![developernotes](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/developernotes/32/1309_2.png) [@developernotes](https://discuss.zetetic.net/u/developernotes)\
**Post date:** [April 1, 2019, 2:45pm UTC](https://discuss.zetetic.net/t/android-api-update/3551/4 "2019-04-01T14:45:40Z")

</div>

Hello @Jeff_Lockhart

Thank you again for your feedback, it is very helpful to hear from users of the library. We will certainly take these into consideration during our review along with the concurrency issue you raised. We will certainly reach out online when we have a beta available for testing.

---

<div class="post-metadata">

**Author:** ![Jeff\_Lockhart](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/jeff_lockhart/32/993_2.png) [@Jeff\_Lockhart](https://discuss.zetetic.net/u/Jeff_Lockhart)\
**Post date:** [April 2, 2019, 4:25am UTC](https://discuss.zetetic.net/t/android-api-update/3551/5 "2019-04-02T04:25:57Z")

</div>

Much appreciated, thank you. Looking forward to this update.

---

<div class="post-metadata">

**Author:** ![greyson-signal](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/greyson-signal/32/1571_2.png) [@greyson-signal](https://discuss.zetetic.net/u/greyson-signal)\
**Post date:** [March 1, 2021, 3:27am UTC](https://discuss.zetetic.net/t/android-api-update/3551/6 "2021-03-01T03:27:17Z")

</div>

@developernotes Are there any plans in the near future to update the locking mechanisms to be more in line with modern android? Jeff brought up great points about performance, and that’s certainly one huge downside about the current locking structure.

However, this isn’t just a performance problem. The way SQLCipher handles locking makes it very easy to deadlock applications. Because beginTransaction() and endTransaction() are actually acquiring/releasing a re-entrant lock under the hood, it means that it’s very easy to accidentally acquire another lock within a transaction and wind-up in a bad situation.

The most common example I can think of is some caching layer over a database. Say you have a cache around a table, with some interface:

```java
public synchronized MyModel get(long id) {
  // if it's in the cache, return it
  // otherwise fetch from DB, cache it, return it
}

```

Now, imagine you accessed that in a transaction:

```java
db.beginTransaction();
// ...
myCache.get(id);
// ...
db.endTransaction();

```

Boom, potential deadlock. It’s possible for Thread A to have the lock on the cache, but unable to get the database lock, because Thread B could have the database lock and is unable to get the lock on the cache.

I have run into several flavors of this issue, and solutions usually revolve around being forced to treat database transactions as your only locking structure, or to do silly dances around accessing other locks inside of transactions.

Android moved away from this current locking approach not just for performance reasons, but for these types of threading issues it would cause. You can see their code comments here:

[https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/android/database/sqlite/SQLiteDatabase.java;l=133?q=sqlitedatabase&ss=android](https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/android/database/sqlite/SQLiteDatabase.java;l=133?q=sqlitedatabase&ss=android)

As a follow-up question: are the changes required to update the API just in java-land, or is there some other underlying issue in native code preventing the API change? [The javadoc for SQLiteDatabase#lockForced()](https://github.com/sqlcipher/android-database-sqlcipher/blob/b2be97e7d42e4ff8b755e88ab641159b00dd3eed/android-database-sqlcipher/src/main/java/net/sqlcipher/database/SQLiteDatabase.java#L588) was a bit concerning.

---

<div class="post-metadata">

**Author:** ![developernotes](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/developernotes/32/1309_2.png) [@developernotes](https://discuss.zetetic.net/u/developernotes)\
**Post date:** [March 1, 2021, 5:41pm UTC](https://discuss.zetetic.net/t/android-api-update/3551/7 "2021-03-01T17:41:07Z")

</div>

Hello @greyson-signal

Thanks for your feedback on the SQLCipher for Android API. We are currently in the process of completing a new SQLCipher release, however, we do plan to review and address the concurrency issues in the Java library (and any potential native underpinnings) following the impending release which will be include in the the following future release.

---

<div class="post-metadata">

**Author:** ![eugene](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.zetetic.net/eugene/32/1603_2.png) [@eugene](https://discuss.zetetic.net/u/eugene)\
**Post date:** [September 21, 2021, 6:49am UTC](https://discuss.zetetic.net/t/android-api-update/3551/8 "2021-09-21T06:49:10Z")

</div>

Hello @developernotes

It’s great that you are looking at concurrency issues. Do you have any ideas when a version with improvements will be released?
