Is UMVUE unique? Is the best unbiased estimator unique?

Suppose $\theta$ is the unknown quantity of interest. A necessary and sufficient condition for an unbiased estimator (assuming one exists) of some parameteric function $g(\theta)$ to be UMVUE is that it must be uncorrelated with every unbiased estimator of zero (assuming of course the unbiased estimator has finite second moment). We can use this result to prove uniqueness of UMVUE whenever it exists.

If possible, suppose $T_1$ and $T_2$ are both UMVUEs of $g(\theta)$.

Then $T_1-T_2$ is an unbiased estimator of zero, so that by the result above we have

$$\operatorname{Cov}_{\theta}(T_1,T_1-T_2)=0\quad,\,\forall\,\theta$$

Or, $$\operatorname{Var}_{\theta}(T_1)=\operatorname{Cov}_{\theta}(T_1,T_2)\quad,\,\forall\,\theta$$

Therefore, $$\operatorname{Corr}_{\theta}(T_1,T_2)=\frac{\operatorname{Cov}_{\theta}(T_1,T_2)}{\sqrt{\operatorname{Var}_{\theta}(T_1)}\sqrt{\operatorname{Var}_{\theta}(T_2)}}=\sqrt\frac{\operatorname{Var}_{\theta}(T_1)}{\operatorname{Var}_{\theta}(T_2)}\quad,\,\forall\,\theta$$

Since $T_1$ and $T_2$ have the same variance by assumption, correlation between $T_1$ and $T_2$ is exactly $1$. In other words, $T_1$ and $T_2$ are linearly related, i.e. for some $a,b(\ne 0)$, $$T_1=a+bT_2 \quad,\text{ a.e. }$$

Taking variance on both sides of the above equation gives $b^2=1$, or $b=1$ ($b=-1$ is invalid because that leads to $T_1=2g(\theta)-T_2$ a.e. on taking expectation, which cannot be true as $T_1,T_2$ do not depend on $\theta$). So $T_1=a+T_2$ a.e. and that leads to $a=0$ on taking expectation. Thus, $$T_1=T_2\quad,\text{ a.e. }$$

The Lehman--Scheffe theorem says the conditional expectation of an unbiased estimator given a complete sufficient statistic is the unique best unbiased estimator.

A complete sufficient statistic for a family of probability distributions is unique in the sense that given the value of any of them, you can compute the value of another without knowing which probability distribution within the family is being sampled from. For example, the pair $$ \left(X_1+\cdots+X_n,X_1^2+\cdots+X_n^2\right) \tag 1 $$ is sufficient for an i.i.d. sample from the family $\{\,N(\mu,\sigma^2) : \mu\in\mathbb R,\ \sigma\in\mathbb R^+\,\}$. So is the pair $$ \left(\bar X = \frac{X_1+\cdots+X_n}{n}, (X_1-\bar X)^2+\cdots+(X_n-\bar X)^2\right) \tag 2 $$ Notice that given the pair $(1)$, you can compute the pair $(2)$ without knowing the values of $\mu$ and $\sigma^2$, and vice-versa. In that sense they are equivalent. Moreover either $(1)$ or $(2)$ is complete in the sense of admitting no unbiased estimator of zero.

Consequently if you find the conditional expectation of any unbiased estimator of any function of $\mu$ and $\sigma^2$ given $(1)$, you get the same thing as if you find the conditional expectation given $(2)$. Hence the UMVUE is unique.