setting font size in XeLaTeX

use the SizeFeature instead of Scale. The last line shows that the two 10pt lines have the same height:

\documentclass[10pt]{article}
\usepackage{fontspec}
\parindent=0pt
\setmainfont{DejaVu Serif}
\setsansfont[Scale=MatchLowercase]{Latin Modern Sans}

\newfontfamily\dejavuseriften[SizeFeatures={Size=10}]{DejaVu Serif}
\newfontfamily\dejavuseriftwelve[SizeFeatures={Size=12}]{DejaVu Serif}

\newfontfamily\lmsansten[SizeFeatures={Size=10}]{Latin Modern Sans}
\newfontfamily\lmsanstwelve[SizeFeatures={Size=12}]{Latin Modern Sans}

\begin{document}
This is a test of DejaVu Serif, as mainfont\\
{\dejavuseriften This is a test of DejaVu Serif, Size=10\\}
{\dejavuseriftwelve This is a test of DejaVu Serif, Size=12\\}

This\textsf{This is a test of Latin Modern Sans, MatchLowercase\\}% compare 
%loercase
{\lmsansten This is a test of Latin Modern Sans, Size=10\\}
{\lmsanstwelve This is a test of Latin Modern Sans, Size=12\\}

\makebox[0pt][l]{This is a test of DejaVu Serif, as mainfont}%
{\lmsansten This is a test of Latin Modern Sans, Size=10\\}

\end{document}