Track changes to lists and dictionaries in python?

I was curious how this might be accomplished when I saw the question, here is the solution I came up with. Not as simple as I would like it to be but it may be useful. First, here is the behavior:

class Tracker(object):
    def __init__(self):
        self.lst = trackable_type('lst', self, list)
        self.dct = trackable_type('dct', self, dict)
        self.revisions = {'lst': [], 'dct': []}


>>> obj = Tracker()            # create an instance of Tracker
>>> obj.lst.append(1)          # make some changes to list attribute
>>> obj.lst.extend([2, 3])
>>> obj.lst.pop()
3
>>> obj.dct['a'] = 5           # make some changes to dict attribute
>>> obj.dct.update({'b': 3})
>>> del obj.dct['a']
>>> obj.revisions              # check out revision history
{'lst': [[1], [1, 2, 3], [1, 2]], 'dct': [{'a': 5}, {'a': 5, 'b': 3}, {'b': 3}]}

Now the trackable_type() function that makes all of this possible:

def trackable_type(name, obj, base):
    def func_logger(func):
        def wrapped(self, *args, **kwargs):
            before = base(self)
            result = func(self, *args, **kwargs)
            after = base(self)
            if before != after:
                obj.revisions[name].append(after)
            return result
        return wrapped

    methods = (type(list.append), type(list.__setitem__))
    skip = set(['__iter__', '__len__', '__getattribute__'])
    class TrackableMeta(type):
        def __new__(cls, name, bases, dct):
            for attr in dir(base):
                if attr not in skip:
                    func = getattr(base, attr)
                    if isinstance(func, methods):
                        dct[attr] = func_logger(func)
            return type.__new__(cls, name, bases, dct)

    class TrackableObject(base):
        __metaclass__ = TrackableMeta

    return TrackableObject()

This basically uses a metaclass to override every method of an object to add some revision logging if the object changes. This is not super thoroughly tested and I haven't tried any other object types besides list and dict, but it seems to work okay for those.

Instead of monkey patching, you can create a proxy class:

• Make a proxy class that inherit from dict/list/set whatever
• Intercept attribute setting, and if the value is a dict/list/set, wrap it into the proxy class
• In proxy class __getattribute__, make sure the method is called on the wrapped type, but take care of tracking before doing so.

Pro:

• no class alteration

Con:

• you are limited to a number of types you know and expect

You could take advantage of the abstract base classes in the collections module, which dict and list implement. This gives you a standard library interface to code against with a short list of methods to override, __getitem__, __setitem__, __delitem__, insert. Wrap the attributes in a trackable adapter inside __getattribute__.

import collections

class Trackable(object):
    def __getattribute__(self, name):
        attr = object.__getattribute__(self, name)
        if isinstance(attr, collections.MutableSequence):
            attr = TrackableSequence(self, attr)
        if isinstance(attr, collections.MutableMapping):
            attr = TrackableMapping(self, attr)
        return attr

    def __setattr__(self, name, value):
        object.__setattr__(self, name, value)
        # add change tracking


class TrackableSequence(collections.MutableSequence):
    def __init__(self, tracker, trackee):
        self.tracker = tracker
        self.trackee = trackee

    # override all MutableSequence's abstract methods
    # override the the mutator abstract methods to include change tracking


class TrackableMapping(collections.MutableMapping):
    def __init__(self, tracker, trackee):
        self.tracker = tracker
        self.trackee = trackee

    # override all MutableMapping's abstract methods
    # override the the mutator abstract methods to include change tracking