Setting up uniform distribution over non-rectangular support

You can use Triangle as a region. I use Region here purely for visualization. It can be omitted.

reg = Region@Triangle[{{0, 0}, {1, 0}, {0, 1}}]

As Roman said, you can apply RegionMoment to it:

RegionMoment[reg, {1, 0}]
(* 1/6 *)

RegionMoment[reg, {1, 0}]/RegionMoment[reg, {0, 0}]
(* 1/3 *)

For more complicated things, you can evaluate any integral over the region:

Integrate[Indexed[x, 1], x \[Element] reg]
(* 1/6 *)

For numerical verification, you can use RandomPoint.

Moment[RandomPoint[reg, 10000], 1]
(* {0.332256, 0.335306} *)

A Dirichlet distribution with parameters $(1,1,1)$ will represent a uniform distribution over exactly the triangle you seek:

ListPlot[
  RandomVariate[DirichletDistribution[{1, 1, 1}], 10000],
  AspectRatio -> Automatic
]

scatter plot of points obtained from that distribution

You can then obtain expectations, moments, etc using the usual statistical machinery in MMA. So for instance, to reproduce the results shown in Roman's answer,

$E(X)$:

Moment[DirichletDistribution[{1, 1, 1}], {1, 0}]     (* Out: 1/3  *)

$E(Y^2)$:

Moment[DirichletDistribution[{1, 1, 1}], {0, 2}]     (* Out: 1/6  *)

$E(XY)$:

Moment[DirichletDistribution[{1, 1, 1}], {1, 1}]     (* Out: 1/12 *)

After some further investigation, it appears I can do what I want by using

dist = ProbabilityDistribution[2 Boole[0 <= x <= 1 && 0 <= y <= 1 - x], {x, 0, 1}, {y, 0, 1}]

Then e.g. $E[X]$, $E[Y^2]$, and $E[XY]$ are given respectively by

Moment[dist, {1, 0}]
(* 1/3 *)

Moment[dist, {0, 2}]
(* 1/6 *)

Moment[dist, {1, 1}]
(* 1/12 *)