Hi folks,
SQLCipher for Android 3.5.4 was just released, it includes two specific changes:
- Fix: address issues with
Cursor::copyStringToBufferthat would previously cause a crash - Fix: convert password bytes from modified UTF-8 to UTF-8
The second item deserves some further explaination. In SQLCipher versons 3.5.0 through 3.5.3, there exists a very rare edge case where a key string value provided to the SQLCipher API would not be encoded properly if it contained an intra-string null character. Null characters (i.e., U+0000) are unlikely to be present in key string values, e.g. they can’t be entered via any standard keyboard, so the vast majority of applications are completely unaffected by this issue. However, if a programatically generated key string were to contain a null, it would result in either an inability to open an existing database, or the creation of a new database that is incompatible with other SQLCipher platforms. It is worth noting that we are only aware of a single instance where this has actually caused a problem [1].
SQLCipher for Android has historically used UTF-8 as the encoding for the keying API within SQLCipher core; this is consistent with the encoding SQLCipher uses on other platforms, too. With the SQLCipher for Android 3.5.0 release, an internal JNI API was used as a replacement for the old ICU-based encoding conversion that was used in the previous version. The new API inadvertently resulted in generating modified UTF-8, which is very similar to stardard UTF-8 except in how it encodes the null character (the former uses two bytes, while the latter uses only one). This was not noticied during pre-release testing because, as mentioned above, null characters in string-based key material are exceedinly rare.
The new version fixes this issue by first attempting to key the database using UTF-8 encoding. If the database can be opened, it will succeed immediately. If the key fails to open the database, and the key also contains a null character, SQLCipher for Android will attempt to open the database using a modified UTF-8 encoded key. If that is successful, the library will automatically and immediately rekey the database using the standard UTF-8 encoded key string so that it is correct going forward.
Both items addressed have coverage included within the SQLCipher for Android test suite to prevent regressions in the future. We welcome your feedback.
[1] In that situation the application was XORing one valid string with another randomly generated string, which could result in both invalid encoding and intra-string zero-bytes where the same character was present in both strings. This is described in detail in the original issue ticket. Thanks to @YuriyWozniak for providing some critical information necessary to identify and resolve the problem.