How much value is given to a recommendation letter written by a foreign professor? (Grad school/math)

The letter being from a foreign university will not itself be a big problem. I think there's always some difficulty switching systems. The most important qualification which might not be obvious (I treat a general high profile and knowing you well as obvious) is that they should be able to convince the reader that they have a good sense that they know what it takes for a student to succeed in the program they are applying to. On average, professors in France are less familiar with US or UK graduate programs than US or UK professors are, so they have a bit less authority on that topic. That's not a huge problem if the letter is otherwise good, though.

I'm actually much more concerned by you saying that they haven't published in a decade. If they really are unknown to the admissions committees that is not good. You might actually ask them if there are any places in the US or UK where they know people well, and where previous students they've written letters for have gone. You don't have to go to those places (though it's worth thinking about), but if the answer really is that there aren't any, that's a big red flag, and you might think a bit harder about the professors at UPMC (which does have a pretty high international profile).