What is an efficient way to Merge two iOS Core Data Persistent Stores?

After several attempts, I've figured out how to make this work. The secret is to first create the incremental store data without any data for the read-only entities. Without leaving read-only data out of the incremental stores, the entities instances for these would get duplicated after the data migration and merge. Hence, the incremental stores should be created without these read-only entities. The default store will be the only store that has them.

For example, I had entities "Country" and "State" in my data model. I needed to have only one instance of Country and State in my object graph. I kept these entities out of incremental stores and created them only in the default store. I used Fetched Properties to loosely link my main object graph to these entities. I created the default store with all the entity instances in my model. The incremental stores either didn't have the read-only entities (i.e., Country and State in my case) to start with or deleted them after data creation is completed.

Next step is to add the incremental store to it's own persistentStoreCoordinator (not the same as the coordinator for the default store that we want to migrate all contents to) during application startup.

The final step is to call migratePersistentStore method on the incremental store to merge its data to the main (i.e., default) store. Presto!

The following code fragment illustrates the last two steps I mentioned above. I did these steps to make my setup to merge incremental data into a main data store to work.

{
    NSError *error = nil;
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
    [NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption, 
    [NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption, nil];

    if (![__persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:defaultStoreURL options:options error:&error])
    {            
        NSLog(@"Failed with error:  %@", [error localizedDescription]);
        abort();
    }    

    // Check for the existence of incrementalStore
    // Add incrementalStore
    if (incrementalStoreExists) {

        NSPersistentStore *incrementalStore = [_incrementalPersistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:incrementalStoreURL options:options error:&error];
        if (!incrementalStore)
        {
            NSLog(@"Unresolved error %@, %@", error, [error userInfo]);
            abort();
        }    

        if (![_incrementalPersistentStoreCoordinator migratePersistentStore:incrementalStore
            toURL:_defaultStoreURL
            options:options
            withType:NSSQLiteStoreType
            error:&error]) 
        {
            NSLog(@"%@", [error userInfo]);
            abort();

        }

        // Destroy the store and store coordinator for the incremental store
        [_incrementalPersistentStoreCoordinator removePersistentStore:incrementalStore error:&error];
        incrementalPersistentStoreCoordinator = nil;
        // Should probably delete the URL from file system as well
        //
    }
}

The reason your migration isn't working is because the managed object model is identical.

Technically, you're talking about "data migration" not "schema migration". CoreData's migration API is designed for schema migration, that is handling changes to the managed object model.

As far as transferring data from one store to another you're kind of on your own. CoreData can help you be efficient by using batching and fetch limits on your fetch requests, but you need to implement the logic yourself.

It sounds like you have two persistent stores, a big one and a small one. It would be most efficient to load the small one and analyze it, discovering the set of primary keys or unique identifiers you need to query for in the larger store.

You could then de-dupe easily by simply querying the larger store for those identifiers.

The documentation for NSFetchRequest has the API for scoping your queries:

https://developer.apple.com/library/mac/#documentation/Cocoa/Reference/CoreDataFramework/Classes/NSFetchRequest_Class/NSFetchRequest.html