Encrypted Coredata with (Merged Model) Migration issue iOS

Anjali · September 12, 2016, 10:13am

DESCRIPTION OF PROBLEM
We have application where the application is not Migrating the data successfully.In the scenario where the data Models are Merged and we have Encrypted Store and there is simple change of addition of new columns in two of the Models.

STEPS TO REPRODUCE
1.Application is already released to client on Enterprise build
2.We are using SQLcipher for using the Encrypted store
3.There are three data Models merged in this Encrypted Store
4.There are two attributes added to the 2 Models which are part on the Encrypted store.
5.During the upgrade the data is not Migrated successfully and hence it is lost.

This may more specific to iOS but is there any known issue around this and also some solution to this ?

wgray · September 12, 2016, 12:55pm

Hi @Anjali,

Encrypted Core Data is a really neat project, but we at Zetetic don’t maintain it or have much familiarity with its internals, though we do keep an eye on it. I’m not sure what the current state of migrations is, but if memory serves, it doesn’t handle all the migration scenarios that Core Data handles. This Issue page on GitHub lists several problems or concerns with doing model migrations in Encrypted Core Data:

github.com/project-imas/encrypted-core-data

Enabling all kinds of migration

project-imas:master ← AndrewMcDrew:migrationRemoval

opened 02:43PM - 13 Jun 16 UTC

AndrewMcDrew

+33 -438

Hi. This pull request enables Core Data migration. It makes it possible to make… non-lightweight migrations, with mapping models and migration policies. The main idea behind it is taken from Apple's [Incremental Store Programming Guide](https://developer.apple.com/library/ios/documentation/DataManagement/Conceptual/IncrementalStorePG/ImplementationStrategy/ImplementationStrategy.html#//apple_ref/doc/uid/TP40010706-CH2-SW2). As a Note in that section states: > Note: `loadMetadata:` should ignore any potential skew between the store and the model in use by the coordinator; this skew is handled automatically later, by the persistent store coordinator. It is sufficient to set the `NSStoreModelVersionHashes` key’s value to the version hashes that were saved in the store metadata the last time the store was saved (if the store is new, you return the version hashes for the current model in use). As far as I understand the process of migration proceeds in this way: 1. `NSPersistentStoreCoordinator` is instantiated with some managed object model, which has it's hashes in the `entityVersionHashesByName` property. 2. While connecting to the actual store this coordinator calls the `loadMetadata:` method. 3. This method sets `NSPersistentStore`'s `metadata` property. A dictionary in this property must contain managed object model's hashes, used the last time with this store. This method do not perform any migrations on it's own, this is a responsibility of the persistent store coordinator. 4. After the `loadMetadata:` method invoking the coordinator checks the `metadata` property. If it's hashes and hashes in this dictionary are different, it starts a migration. 5. During this migration a whole new Core Data stack is set up. This stack includes a new instance of `NSPersistentStore` class. For this instance the situation looks like it is it's first launch, even though it's managed object model is a new one. During the migration all of the old managed objects retrieved from the old store, passed through migration rules (mapping model, either defined by developer or inferred automatically as a part of a lightweight migration), and inserted into a new stack. From the new stack's persistent store point of view this looks the same as a just new installation. So the problem was in the migration code in the Encrypted Core Data's `loadMetadata` method. I removed it and this gave Core Data a possibility to follow it's procedure, including any kind of migration. Additionally, I removed all of the code, that is related to the migration. Maybe it's to radical. There is one more technical problem here. To make a migration work it is necessary to store that hashes properly in the `metadata` property. This property must be overriden due to `NSPersistentStore` doc, but `NSIncrementalStore` overrides it by itself, so that there is no need to do that. Unfortunately, it does it in some strange way. It looks like this: the persistent store properly retrieves it's hashes, set them to the `metadata` property, but after the setting there are a new hashes in that property! This is the reason why the migration doesn't start in this situation -- Core Data doesn't see a difference in hashes. So I've overriden the `metadata` property, to preserve the hashes and to keep everithing else in the metadata untouched. The latter is important, because this `NSIncrementalStore`'s setter additionally sets some internal keys and values in the dictionary. Without this info the migration doesn't start either. Currently this is Apple's bug as for me, so I plan to post a bug to them regarding this property behaviour and look what they will answer. Additionally, I've noticed a lot of tests in the example project. I am sorry but I haven't appropriate knowledge regarding tests. In fact, I wasn't even able to ran this example project -- I've done all of the job in my project, in which ECD is added via Cocoapods. So I haven't touched tests. The last problem -- I am not aware whether this custom migration in the `loadMetadata` method allowed migration from an unencrypted store to the encrypted one. If so -- this change brakes it. But I believe this is not necessary for all of the Encrypted Core Data users -- if for the first app's users their stores already encrypted, then main priority is to make the migration just for an encrypted store. This change can be useful for such issues: #190, #14, #141, #83.

I’m not sure if any of the issues there are related to your problem but it might be worth a look. Do you know what version of SQLCipher your are working with? The project authors have left a note in the README file that it doesn’t support migrating from SQLCipher 2 to SQLCipher 3 databases:

Along with the move to CommonCrypto, we’ve updated the version of SQLCipher included as a submodule from v2.0.6 to v3.1.0. Databases created with v2.0.6 will not be able to be read directly by v3.1.0, and support for legacy database migration is not yet supported by ECD.

Topic		Replies	Views
Migration Old unencrypted Core database to Encrypted Core Data base(using SQLCipher) SQLCipher	12	1778	August 21, 2020
SQLCipher Migration From 3.4.1 to 4x in iOS SQLCipher	3	2407	April 24, 2019
SQLCipher iOS : issue in old DB migration Issues	1	1181	February 13, 2015
Updating an iOS application with SQLCipher included version SQLCipher	3	872	October 20, 2016
Terminating app due to uncaught exception SQLCipher	5	605	February 2, 2022

Encrypted Coredata with (Merged Model) Migration issue iOS

Related topics