Migrating from CommonCrypto to OpenSSL


#1

Hello Zetetic Community,

I find myself in an interesting situation and was wondering if I could get some advice on how to overcome what I’ve been struggling with.

Our iOS apps have been using SQLCipher (3.8.4.3) CommonCrypto CBC for a while and my recent task has been to switch CommonCrypto with OpenSSL and CBC with GCM. I’ve managed to get SQLCipher updated to 3.15.2 and building successfully with OpenSSL and have verified that it can switch to GCM without issue. The issue I’m having lies within trying to open the CommonCrypto db files that are in our apps using the updated SQLCipher using OpenSSL. Essentially we want to migrate the db file to GCM using ATTACH and the sqlcipher_export() function but the ATTACH statement results in the error “File is encrypted or not a database” even though keying succeeds or at least fails silently.

I’ve done some research before posting this and suspect I need to alter some PRAGMA settings in order to get the CommonCrypto db files to be readable but so far that avenue hasn’t resulted in any progress. Since both versions of SQLCipher used were within 3.x.x.x I don’t think I need to worry about file format and don’t have to use cipher_migrate(). Is that incorrect? Are CommonCrypto and OpenSSL db files incompatible with each other or is there a way to allow one to be used in the other?

-Brad


#2

I’d appreciate any feedback on this topic. I can provide more details if needed.


#3

Hi @bcatlett

Thanks for your interest in SQLCipher and for posting on the discussion forum.

Essentially we want to migrate the db file to GCM using ATTACH and the sqlcipher_export() function but the ATTACH statement results in the error “File is encrypted or not a database” even though keying succeeds or at least fails silently.

You might have a look here (if you haven’t already): https://www.zetetic.net/sqlcipher/sqlcipher-api/#sqlcipher_export under Example 4 which outlines a scenario of changing cipher settings.

Please note: PRAGMA cipher is deprecated. In other words, this might work for you now, but the feature may be modified or removed from a future version of SQLCipher.

Since both versions of SQLCipher used were within 3.x.x.x I don’t think I need to worry about file format and don’t have to use cipher_migrate(). Is that incorrect?

That’s correct, you cannot use cipher_migrate() here.

Are CommonCrypto and OpenSSL db files incompatible with each other or is there a way to allow one to be used in the other?

I don’t believe (but I’m not certain) that CommonCrypto provides all the ciphers that OpenSSL does, but in the default configuration they are compatible. When creating the previous database, had you adjusted any of the settings from the default (i.e. kdf iterations/page size)?

It would be helpful to post an example of the code that’s failing for you.

Cheers,
Micah


#4

Hello Micah,

Thank you for replying. I appreciate you feedback.

Your are correct, CommonCrypto doesn’t provide the aes-256-gcm cipher which is why we went to OpenSSL. The previous database was encrypted using the default SQLCipher 3.8.4.3 configuration using CommonCrypto and aes-256-cbc and since we wanted to move to aes-256-gcm we updated our version of SQLCipher to 3.15.2, configured it to use OpenSSL, and use the default configuration except for changing the cipher to gcm.

I’ll include the piece of code in question. It is basically a objc version of the sqlcipher_export() documentation link you included in your post. The oldEncrypt_DB reference is our cbc encrypted database in this scenerio. The newDBPath is created somewhere above this code.

Fails on Step 3. Attached old database with the error stating “File is encrypted or not a database”. I’ve also attempted to make sure the default cipher is still cbc but when I attempted to execute “PRAGMA Cipher” after opening and keying the database, it returns SQLITE_DONE. That is what got me thinking that either CommonCrypto files are incompatible with OpenSSL files or I need to adjust some configuration settings after opening the previous database file to get this to work.

Again Thank you for replying,

-Brad

NSString * filepathString = [self dataFilePath];
const char *filepath_param = [filepathString UTF8String];
if ([[NSFileManager defaultManager] fileExistsAtPath:filepathString]) {
    if (sqlite3_open(filepath_param, &oldEncrypted_DB) != SQLITE_OK)
    {
        sqlite3_close(oldEncrypted_DB);
    }
}

/* 2. Provide Key */
NSString *sql_pragma_key = [NSString stringWithFormat:@"PRAGMA key = '%s';",[FIPS_SQLKey() UTF8String]];
const char *parm_pragmaKey = [sql_pragma_key UTF8String];
int results_pragmaKey = sqlite3_exec(oldEncrypted_DB, parm_pragmaKey, NULL, NULL, NULL);
if ( results_pragmaKey != SQLITE_OK)
{
    NSLog(@"pragma key error: %d",results_pragmaKey);
}

/* 3. Attached old database */
NSString *sql_attach = [NSString stringWithFormat:@"ATTACH DATABASE '%s' AS encrypted KEY '%s';",[newDBPath UTF8String], [FIPS_SQLKey() UTF8String]];
char *error;
const char *parm_attach = [sql_attach UTF8String];
int results_attach = sqlite3_exec(oldEncrypted_DB, parm_attach, NULL, NULL, &error);
if ( results_attach != SQLITE_OK)
{
    NSLog(@"Something went wrong with creating or attaching the encrypted database: %d, %s",results_attach, error);
}

/* 4. Updating Cipher */
NSString *sql_cipherMode = @"PRAGMA encrypted.cipher='aes-256-gcm';";
const char *parm_cipherMode = [sql_cipherMode UTF8String];   // This needs to be UTF8String
int results_cipherMode = sqlite3_exec(oldEncrypted_DB, parm_cipherMode, NULL, NULL, NULL);
if ( results_cipherMode != SQLITE_OK)
{
    NSLog(@"error adding cipher mode: %d",results_cipherMode);
}

/* 5. Export unencrypted DB to new encrypted DB */
const char *parm_sqlcipher_export = "SELECT sqlcipher_export('encrypted');";
int results_sqlcipher_export = sqlite3_exec(oldEncrypted_DB, parm_sqlcipher_export, NULL, NULL, NULL);
if (results_sqlcipher_export != SQLITE_OK)
{
    NSLog(@"un-able to export database: %d",results_sqlcipher_export);
}

/* 6. detaching database */
int results_detaching = sqlite3_exec(oldEncrypted_DB, "DETACH DATABASE encrypted;", NULL, NULL, NULL);
if (results_detaching != SQLITE_OK)
{
    NSLog(@"error detaching database: %d",results_detaching);
}

/* 7. Closing Database */
sqlite3_close(oldEncrypted_DB);

#5

@bcatlett It is not really clear from the example what the default cipher is set to when you build sqlcipher. Since you are migrating a CommonCrypto database, the cipher for that database will be aes-256-cbc.

If you have changed the default cipher to aes-256-gcm (i.e. via define while compiling), you must override that for the initial connection to the old database using PRAGMA cipher = ‘aes-256-cbc’ against oldEncryptedDb. Then you can attach the export database, set the cipher there for aes-256-gcm, and finally run the export.

If you have not changed the default cipher, so it is still aes-256-cbc, then it looks like the code you posted should basically work. However, in order for it to work you need to make sure that there is no file in existence at newDBPath. In other words, you should be allowing the attach to create a brand new database, not specifying the path to an existing database. In other words, calling PRAGMA encrypted.cipher = ‘aes-256-gcm’ will not really update a cipher as mentioned in the #4 comment, it will only set the cipher when initializing a new database.


#6

Thank you @mmoore and @sjlombardo for replying. Your responses gave me the idea to make sure that the app using sqlcipher with the old database was properly configured. Turns out that “libsqlite3.0.dylib” was being used thus causing my issues now. Once that was removed from the older project, the migration to GCM works fine.

I am encountering an issue that I don’t think is related to this but it may. I’m encountering a “invalid bitcode signature” error now when I run the project using our updated sqlcipher on anything except a simulator. This to me seems like an OpenSSL build issue I need to look into but if anyone has encountered this with SQLCipher in their projects before has any advice, that of course would be appreciated.

Thanks again @mmoore & @sjlombardo,

-Brad


#7

@bcatlett

Glad to hear you were able to get the migration to work.

I’m encountering a “invalid bitcode signature” error now when I run the project using our updated sqlcipher on anything except a simulator

You may want to verify that your project Build Settings > Architectures > Build Active Architecture Only > Debug is set to YES.

Cheers,
Micah