Typesetting Right-to-Left math
As promised in the comment above, the new version of the XITS font has preliminary RTL support. There is a simple ConTeXt test file in the repository (you need a recent ConTeXt MkIV version) but it shouldn't be hard to port it to LuaLaTeX and unicode-math package.
It is work in progress, not all symbols have been mirrored and support for the proposed Arabic math symbols still lacking.
There is one caveat though, luatex reverses the direction of everything including numbers, so you either want to do something like {\textdir TLT 123} or try my experimental luadirections package (however it doesn't work in math mode yet, at least in ConTeXt, but I'm investigating that.)
The following is not a direct answer to the question, since it is more focused on Unicode than on TeX.But I think it is relevant nevertheless, specially with XeLaTeX, and has probably to be kept in mind if someone want to write a package for Arabic mathematical typesetting.
Unicode has encoded unicode 6.1 ( January 2012) a set of Arabic Mathematical Symbols in for at positions 1EE00..1EEF1 (pdf chart). Some fonts will probably include them.
Some other are encoded since unicode 5.1 and are present in some fonts. They are :
- ؆ U+0606 ARABIC-INDIC CUBE ROOT
- ؇ U+0607 ARABIC-INDIC FOURTH ROOT
- ؈ U+0608 ARABIC RAY
- ؉ U+0609 ARABIC-INDIC PER MILLE SIGN
- ؊ U+060A ARABIC-INDIC PER TEN THOUSAND SIGN
- U+2B30 — U+2B4C a set of reversed arrows for RTL maths. See also the discussion in n3259 (pdf) for the use of reverse vs mirror arrows.
The final proposal to unicode, with some background to the character repertoire is n3799 (pdf), and the original proposals from 2006, which have more details on the specific typographic for Arabic math (like a special aleph to avoid confusion with 1 and so on) is n3085-1 (pdf) and n3086-1(pdf).
Azzeddine Lazrek, from Cadi Ayyad University (Marrakech) is at the origin of these proposals, and has webpage which contains some LaTeX packages. This webpage als an example comparing the typesetting of the same mathematical expression using various typographic traditions (English, French, Moroccan, Western Arabic and Eastern Arabic).