How to add LateX/Math to Windows 10 Ink API?

Yes for the first point, you can convert ink strokes to text using handwriting recognition.

Recognize ink strokes

For math and equations, didn't see any default Recognizer at the moment.

-Understand handwriting and digitize it?

Yes, that's the whole point of the API.

It's basically OCR 2.0

-Understand Math/Equations and digitize it (ie Latex)?

Nope. Or, at least, not yet because, of course, it would be a logical expansion of the API.

Are there any other APIs that can do this for UWP or Unity?

Not that I had heard of.

Finally, I don't think that Jeans link is what you are looking for. It's not simply a matter of adding a new symbol, but also to represent it conveniently.

For example, how would you do to display \frac{\,\frac{a}{b}\,}{\frac{c}{d}} ?

There is no correct way of positioning the a/b over the fraction bar and the c/d under it. Even if you can use superscript and subscript, there is no way for a standard police to align them vertically.

The good news is: Ink is an API, which means you could create a wrapper that would, for example, automate the latex pdf generation. So yes, you could create the functionality you're looking for with a bit on C# (as it looks you're on a Microsoft environment)

Take a look at this thread: https://tex.stackexchange.com/questions/53914/compile-tex-inside-microsoft-c-net

Best of luck!