Unpickling python objects with a changed module path

In addition to @MartinPieters answer the other way of doing this is to define the find_global method of the cPickle.Unpickler class, or extend the pickle.Unpickler class.

def map_path(mod_name, kls_name):
    if mod_name.startswith('packageA'): # catch all old module names
        mod = __import__('WrapperPackage.%s'%mod_name, fromlist=[mod_name])
        return getattr(mod, kls_name)
    else:
        mod = __import__(mod_name)
        return getattr(mod, kls_name)

import cPickle as pickle
with open('dump.pickle','r') as fh:
    unpickler = pickle.Unpickler(fh)
    unpickler.find_global = map_path
    obj = unpickler.load() # object will now contain the new class path reference

with open('dump-new.pickle','w') as fh:
    pickle.dump(obj, fh) # ClassA will now have a new path in 'dump-new'

A more detailed explanation of the process for both pickle and cPickle can be found here.

One possible solution is to directly edit the pickle file (if you have access). I ran into this same problem of a changed module path, and I had saved the files as pickle.HIGHEST_PROTOCOL so it should be binary in theory, but the module path was sitting at the top of the pickle file in plain text. So I just did a find replace on all of the instances of the old module path with the new one and voila, they loaded correctly.

I'm sure this solution is not for everyone, especially if you have a very complex pickled object, but it is a quick and dirty data fix that worked for me!

You'll need to create an alias for the pickle import to work; the following to the __init__.py file of the WrapperPackage package:

from .packageA import * # Ensures that all the modules have been loaded in their new locations *first*.
from . import packageA  # imports WrapperPackage/packageA
import sys
sys.modules['packageA'] = packageA  # creates a packageA entry in sys.modules

It may be that you'll need to create additional entries though:

sys.modules['packageA.moduleA'] = moduleA
# etc.

Now cPickle will find packageA.moduleA and packageA.moduleB again at their old locations.

You may want to re-write the pickle file afterwards, the new module location will be used at that time. The additional aliases created above should ensure that the modules in question have the new location name for cPickle to pick up when writing the classes again.