Why don't (can't) astronomers take advantage of interferometry to the extreme?

Interferometry can and has been conducted with intercontinental baselines. This is how the Event Horizon Telescope works - but at microwave wavelengths. However, at optical wavelengths the longest baselines are of order 100 metres (e.g. at the VLT site in Paranal and the Keck telescopes on Mauna Kea).

Interferometry at infrared and shorter wavelengths is more difficult than at microwave/radio wavelengths for a number of reasons (see Monnier 2003). Radio signals can basically be recorded on tape (or rather hard drives these days) at different sites and then recombined (or correlated) "off-line" at another location. This won't work at optical wavelengths because of the higher frequencies. Data cannot yet be recorded at this rate and the storage problems for such data would be enormous. Instead, optical interferometers implement a hardware solution - they recombine the signals from different telescopes by sending the light along optical delay lines that compensate for the separations of the telescopes, before bringing the signals together directly to form the interference patterns. There are multiple hard problems to be solved with this approach.

Visible light is badly affected by the atmosphere. This introduces phase errors for telescopes situated in different places. The phase error is not the same for objects that differ in position by only a few arcseconds (the so-called "isoplanatic patch"), so large scale imaging is not possible. This latter point also prevents the "phase referencing" technique used in radio interferometers where any phase noise is calibrated out by periodically looking at another nearby (bright) reference source that is within the isoplanatic patch.

It is possible to use the "closure phase" technique in optical interferometry. The phase errors introduced in one baseline $e_1 - e_2$, can be eliminated by combining the signals within a triplet of baseline: $(e_1 - e_2) + (e_2 - e_3) + (e_3 - e_1) = 0$. This works if the sources are bright enough that sufficient signal can be obtained in the time it takes the pathlength introduced by the atmosphere to change. At optical wavelengths this can be as short as 10-20 ms, and is much shorter than the radio coherence time. You might think then that just using large telescopes would help boost the signal strength, but unfortunately the coherence length of atmospheric turbulence (the transverse distance over which significant pathlength differences are expected) means that apertures greater than about 10cm don't really improve matters unless adaptive optics systemas are also used.

A further problem is that in order to observe faint objects you would like to observe over a broad wavelength bandwidth. But unless one restricts the bandwidth to a small fraction of the observation wavelength, then a different delay line length is required for sources viewed at slightly different angles. Imagine trying to do Young's double slit experiment using light that isn't monochromatic - the fringes will appear in slightly different places according to their wavelength. This amounts to a chromatic aberration. You basically have a trade-off between a very narrow field of view and bandwidth.

This puts severe constraints on the optical pathlengths used in the array of telescopes - basically you end up with requiring the various pathlengths between the telescopes and where the signals are recombined, to be the same within a wavelength of light and this precison is difficult to achieve over longer baselines. For instance you need very precisely controlled delay lines running in precisely measured vacuum tubes. What's more, because of the Earth's rotation, then to keep pathlengths similar as an object moves with respect to the telescope array, then the delay lines need fast, but accurate, moving components to compensate for this!

i.e. It is not just that you have to make the various pathlengths similar to within a wavelength of light; you have to keep them that way with moving parts/mirrors etc. The picture below shows the "Paranal Express", which is an optical platform that moves (at up to 50 cm/s) to compensate for the "sidereal optical path difference" at the VLT Interferometer in Chile.

Although in principle an interferometer can achieve the same angular resolution as a telescope with a diameter similar to the longest baseline it will never be as sensitive. The E-ELT will act as a big photon-collecting bucket that cannot be matched even by a large number of widely spaced smaller telescopes, especially when considering the limitations on aperture and integration time discussed above. Optical interferometry is restricted to observing very bright targets at present. There are a limited number of photons present in the "coherent volume of atmosphere" above the telescope (the coherent volume here is the square of the coherence length multiplied by the speed of light and the atmospheric coherence time).

Lastly it has to be said that interferometers cannot create a perfect image from a sparsely sampled array. What they detect are interference patterns and there are complex algorithms that then attempt to reconstruct the astronomical image that led to the interference pattern. These algorithms can often yield ambiguous, or certainly non-unique, results. So far, at optical wavelengths, most work has been limited to simple measurements of the diameters of stars and separations of binary systems, or other simple geometric arrangements.