Integration by parts for general measure?

As you have pointed out there is no reason that an arbitrary measure should preserve this property so when some sort of integration by parts result holds that should be quite a remarkable feature, and when these measures are normalisable you are in fact making statements about a relationship between the expected value of a function and the expectation of its derivatives.

As is so often the case it is illuminating to work with the Gaussian distribution, so let $\varphi$ be the standard Gaussian measure then it is true that $$ \mathcal{A}\varphi := \varphi^{\prime}(x) + x\varphi(x) = 0, $$ where $\mathcal{A}$ is the operator $\frac{d}{dx} + x$ with adjoint $\frac{d}{dx} - x$ (where this adjoint is formed from integration by parts with respect to the Lebesgue measure. Now let $f$ be a function such that all of the following operations are legal and so on, and let $ \rho(dx)$ be the measure $\varphi(x) dx $. Then we have $$ \int (A^{\dagger} f )(x) \rho (dx) = \int f(x) \mathcal{A} \varphi(x) dx = 0. $$ So we get a version of integration by parts for the Gaussian measure with the operator $d/dx$ is $$ \begin{align} \int f^{\prime} g d\rho &= \int (fg)^{\prime} - g^{\prime} f d\rho \\ &= \int x f g d\rho - \int g^{\prime} f d\rho. \end{align} $$ Whether this is useful is of course going to be problem dependent. In your original question you made this term disappear by assuming a compact support, and of course that is the same thing you would do to make the standard Lebesgue measure proportional to the uniform distribution. In the probability and statistics literature the process of using these operators to prove limit theorems and other approximation type results is referred to as Stein's method, this blog post gives an introduction and application to the CLT https://terrytao.wordpress.com/2010/01/05/254a-notes-2-the-central-limit-theorem/

You are correct. A measure has no reason to preserve this property. As a matter of fact, the only measures that absolutely continuous wrt Lebesgue measure that preserve this property for all $f,g \in C^1_c(\Bbb{R})$ are scaled Lebesgue measures.

Let $g\neq 0$. If $(gh)'=g'h$, then $gh'+g'h=g'h$, or $ gh'=0$, implying $h'=0$ and $h=c$.

Or if the question is, "given two specific functions, $f,g$, does there exist $\mu$ such that this formula holds?". The answer is only if $f$ or $g$ is identically $0$.