matplotlib linked x axes with autoscaled y axes on zoom

After studying the gory details of matplotlib's axes.py, it appears that there are no provisions to autoscale an axes based on a view of the data, so there is no high-level way to achieve what I wanted.

However, there are 'xlim_changed' events, to which one can attach a callback:

import numpy as np

def on_xlim_changed(ax):
    xlim = ax.get_xlim()
    for a in ax.figure.axes:
        # shortcuts: last avoids n**2 behavior when each axis fires event
        if a is ax or len(a.lines) == 0 or getattr(a, 'xlim', None) == xlim:
            continue

        ylim = np.inf, -np.inf
        for l in a.lines:
            x, y = l.get_data()
            # faster, but assumes that x is sorted
            start, stop = np.searchsorted(x, xlim)
            yc = y[max(start-1,0):(stop+1)]
            ylim = min(ylim[0], np.nanmin(yc)), max(ylim[1], np.nanmax(yc))

        # TODO: update limits from Patches, Texts, Collections, ...

        # x axis: emit=False avoids infinite loop
        a.set_xlim(xlim, emit=False)

        # y axis: set dataLim, make sure that autoscale in 'y' is on 
        corners = (xlim[0], ylim[0]), (xlim[1], ylim[1])
        a.dataLim.update_from_data_xy(corners, ignore=True, updatex=False)
        a.autoscale(enable=True, axis='y')
        # cache xlim to mark 'a' as treated
        a.xlim = xlim

for ax in fig.axes:
    ax.callbacks.connect('xlim_changed', on_xlim_changed)

Unfortunately, this is a pretty low-level hack, which will break easily (other objects than Lines, reversed or log axes, ...)

It appears not possible to hook into the higher level functionality in axes.py, since the higher-level methods do not forward the emit=False argument to set_xlim(), which is required to avoid entering an infinite loop between set_xlim() and the 'xlim_changed' callback.

Moreover, there appears to be no unified way to determine the vertical extent of a horizontally cropped object, so there is separate code to handle Lines, Patches, Collections, etc. in axes.py, which would all need to be replicated in the callback.

In any case, the code above worked for me, since I only have lines in my plot and I am happy with the tight=True layout. It appears that with just a few changes to axes.py one could accommodate this functionality much more elegantly.

Edit:

I was wrong about not being able to hook into the higher-level autoscale functionality. It just requires a specific set of commands to properly separate x and y. I updated the code to use high-level autoscaling in y, which should make it significantly more robust. In particular, tight=False now works (looks much better after all), and reversed/log axes shouldn't be a problem.

The one remaining issue is the determination of the data limits for all kinds of objects, once cropped to a specific x extent. This functionality should really be built-in matplotlib, since it may require the renderer (for example, the code above will break if one zooms in far enough that only 0 or 1 points remain on screen). The method Axes.relim() looks like a good candidate. It is supposed to recalculate the data limits if the data have been changed, but presently handles only Lines and Patches. There could be optional arguments to Axes.relim() that specify a window in x or y.