Efficient way of averaging over elements of one matrix based on another

Here is one idea using Pick and Total. Setup:

SeedRandom[123];
n = 3000;
nedges = 12000;
g = RandomGraph[{n,nedges}];

distmatrix=GraphDistanceMatrix[g];

m = RandomReal[{0.1,1}, {n, n}];
m = UpperTriangularize[m, 1]+Transpose[UpperTriangularize[m, 1]];
ones = ConstantArray[1, {n, n}];

distances = Union @ Flatten @ distmatrix

{0, 1, 2, 3, 4, 5, 6, 7}

Now, using Pick and Total to find the averages for 6:

Total[Pick[m, distmatrix, 6], Infinity]/Total[Pick[ones, distmatrix, 6], Infinity] //AbsoluteTiming

{0.328108, 0.550647}

A table of the values for the positive distances:

Table[Total[Pick[m, distmatrix, i], Infinity]/Total[Pick[ones, distmatrix, i], Infinity], {i, Rest @ distances}] //AbsoluteTiming

{1.93576, {0.548076, 0.548365, 0.549827, 0.549896, 0.550361, 0.550647, 0.527063}}

---

Another possibility is to create a pair matrix and use GroupBy. To be most efficient, though, the distances should be reals. So:

pairs = Flatten[{N @ distmatrix, m}, {{2,3}, {1}}]; //AbsoluteTiming

{0.305517, Null}

creates a list of {d, v} pairs. Then using GroupBy:

r2 = KeySort @ KeyMap[Round] @ GroupBy[pairs, First -> Last, Mean]; //AbsoluteTiming
Rest @ Values @ r2

{1.58506, Null}

{0.548076, 0.548365, 0.549827, 0.549896, 0.550361, 0.550647, 0.527063}

which is the same as before.

Addendum

Note that the GroupBy approach can be sped up a bit further by making use of the "GroupByList" ResourceFunction:

ResourceFunction["GroupByList"][Flatten @ m, Flatten @ distmatrix, Mean] //AbsoluteTiming

{0.148771, <|0 -> 0., 4 -> 0.549896, 5 -> 0.550361, 3 -> 0.549827, 2 -> 0.548365, 6 -> 0.550647, 1 -> 0.548076, 7 -> 0.527063|>}

---

An even faster possibility is to work with vectors instead (modified from earlier answer by removing the unnecessary Unitize)

fd = Flatten @ distmatrix;
fm = Flatten @ m;

Table[
    With[{v = Clip[fd, {i, i}, {0, 0}]},
        v . fm / Total[v]
    ],
    {i, Rest @ distances}
] //AbsoluteTiming

{0.445384, {0.548076, 0.548365, 0.549827, 0.549896, 0.550361, 0.550647, 0.527063}}