What algorithm is behind the Gimp's "Color to Alpha" feature?

I took a look at the source code, and the meat of it is the colortoalpha function. The parameters *a1 to *a4 are the input/output red, green, blue and alpha, respectively, and c1 to c3 is the color to make alpha.

When you're combining two colors c1 and c2 with a specific alpha a (0 ≤ a ≤ 1), the result is

y = a * c1 + (1-a) * c2

Here we're doing the reverse operation: We know the end result y and the background color c2, and want to figure out c1 and a. Since this is an under-specified equation, there's an infinite amount of solutions. However, the ranges 0 ≤ c1 ≤ 255 and 0 ≤ a ≤ 1 adds bounds to the solution.

The way the Gimp plugin works is that for each pixel it minimizes the alpha value (i.e. maximizes transparency). Conversely, this means that for each resulting pixel that isn't completely transparent (i.e. was not exactly the background color), one of the RGB components is either 0 or 255.

This produces an image that when overlayed on top of the specified color will produce the original image (in absence of rounding errors) and has maximum transparency for each pixel.

It's worth noting that the whole process is done in the RGB color space, but could be performed in others as well, as long as the combining operation is done in the same color space.