Using xetex, I'd like to set the monospace font size for my entire text

Use the Scale attribute on \setmonofont, e.g.:

\setmonofont[Scale=MatchLowercase]{Droid Sans Mono}
\setmonofont[Scale=0.9]{Droid Sans Mono}

The scale can be:

• a number: 1 is the original size, 2 is twice as big, 0.5 is twice as small...
• MatchLowercase or MatchUppercase: this scales the font to the height of either the lowercase or the uppercase letters in the current roman font.

Source: Section 6.2 in the manual of the fontspec package (PDF)

In manual of fontspec: 7.2 Scale

http://www.ctan.org/tex-archive/macros/latex/contrib/fontspec