python - recursively deleting dict keys?

Adding or removing items to/from a sequence while iterating over this sequence is tricky at best, and just illegal (as you just discovered) with dicts. The right way to remove entries from a dict while iterating over it is to iterate on a snapshot of the keys. In Python 2.x, dict.keys() provides such a snapshot. So for dicts the solution is:

for key in mydict.keys():
    if key == bad_value:
        del mydict[key]

As mentionned by cpizza in a comment, for python3, you'll need to explicitely create the snapshot using list():

for key in list(mydict.keys()):
    if key == bad_value:
        del mydict[key]

For lists, trying to iterate on a snapshot of the indexes (ie for i in len(thelist):) would result in an IndexError as soon as anything is removed (obviously since at least the last index will no more exist), and even if not you might skip one or more items (since the removal of an item makes the sequence of indexes out of sync with the list itself). enumerate is safe against IndexError (since the iteration will stop by itself when there's no more 'next' item in the list, but you'll still skip items:

>>> mylist = list("aabbccddeeffgghhii")
>>> for x, v  in enumerate(mylist):
...     if v in "bdfh":
...         del mylist[x]
>>> print mylist
['a', 'a', 'b', 'c', 'c', 'd', 'e', 'e', 'f', 'g', 'g', 'h', 'i', 'i']

Not a quite a success, as you can see.

The known solution here is to iterate on reversed indexes, ie:

>>> mylist = list("aabbccddeeffgghhii")
>>> for x in reversed(range(len(mylist))):
...     if mylist[x] in "bdfh":
...         del mylist[x]
>>> print mylist
['a', 'a', 'c', 'c', 'e', 'e', 'g', 'g', 'i', 'i']

This works with reversed enumeration too, but we dont really care.

So to summarize: you need two different code path for dicts and lists - and you also need to take care of "not container" values (values which are neither lists nor dicts), something you do not take care of in your current code.

def scrub(obj, bad_key="_this_is_bad"):
    if isinstance(obj, dict):
        # the call to `list` is useless for py2 but makes
        # the code py2/py3 compatible
        for key in list(obj.keys()):
            if key == bad_key:
                del obj[key]
            else:
                scrub(obj[key], bad_key)
    elif isinstance(obj, list):
        for i in reversed(range(len(obj))):
            if obj[i] == bad_key:
                del obj[i]
            else:
                scrub(obj[i], bad_key)

    else:
        # neither a dict nor a list, do nothing
        pass

As a side note: never write a bare except clause. Never ever. This should be illegal syntax, really.