For a unitary matrix $U$, what's the minimal value of the real part of $\det(U^*)\prod_i U_{ii}$?

We consider the real case.

$\textbf{Proposition}$. Let $n>2$ and

$f:U=[u_{i,j}]\in O(n)\mapsto \det(U^T)\Pi_{i=1}^n u_{i,i}$.

Then the minimum of $f$ is $m=-(1-2/n)^n$.

$\textbf{Proof}$. Since $O(n)$ is compact, the lower bound of $f$ is reached in at least a matrix $A=[a_{i,j}]\in O(n)$. Note that if we change a column of $U\in O(n)$ into its opposite, then the obtained matrix $U'\in O(n)$ satisfies $f(U)=f(U')$.

Consequently, we may assume that, for every $i$, $a_{i,i}\geq 0$. Since we know, from the OP, that $m\leq -(1-2/n)^n< 0$, we deduce that, for every $i$, $a_{i,i}>0$ and $\det(A)<0$.

Then $-1\in spectrum(A)$ and $\sum_i a_{i,i}=trace(A)\leq n-2$.

Consequently, $0<\Pi_i a_{i,i}\leq (\dfrac{n-2}{n})^n$ (fixing the sum of the $(a_i)$ in $n-2$, the max of the product is reached when the $(a_{i,i})$ are equal)

and we are done. $\square$

Let $n>2$. Define $f:M_n(\mathbb{C})\to\mathbb{R}$ with $f(X)=\operatorname{Re}\left(\det(X^*)\prod_{k=1}^nx_{kk}\right)$.

Note that $f(X) = f(X^T) = f(\overline{X}) = f(X^*) = f(PXP^T) = f(XD)$ for every permutation matrix $P$ and diagonal unitary matrix $D$.

Function $f$ is continuous and therefore obtains its minimum on the set of unitary matrices. Let $U$ be a matrix for which minimum $m=f(U)<0$ is attained. We may assume $u_{kk} > 0$, otherwise we note that $f(U)<0$ implies diagonal elements are non-zero and we can multiply each column $k$ with $|u_{kk}|/u_{kk}$ to achieve our assumption. Let $\zeta=-\det{U^*}$ and note that $f(U)<0$ implies $\operatorname{Re}(\zeta)>0$.

We are going to show that $U$ is also a Hermitian matrix. To obtain relation between its off-diagonal elements we will exploits the fact that $f(UQ)\geq f(U)$ for every unitary $Q$.

Let $i,j\in\{1,\ldots,n\}$, $i\neq j$ be arbitrary. For these $i$ and $j$ we define a unitary matrix $Q(\varphi)$ as a matrix obtained from identity by replacing submatrix at the intersection of rows and columns $i$ and $j$ with $$\begin{bmatrix}1&0\\0&\xi\end{bmatrix}\begin{bmatrix}\cos\varphi & -\sin\varphi\\\sin\varphi & \cos\varphi\end{bmatrix}\begin{bmatrix}1&0\\0&\overline{\xi}\end{bmatrix}\,,$$ where $\xi$ is unimodular number such that $u_{ij}\xi=|u_{ij}|$. We note that $\det(UQ(\varphi))=\det(U)$ and that diagonal of $U$ and $UQ(\varphi)$ differ only at positions $(i,i)$ and $(j,j)$.

Now, we define function $g:\mathbb{R}\to\mathbb{R}$ with $g(\varphi)=f(UQ(\varphi))$. Using previous results, we have \begin{align} g(\varphi) &=f(UQ(\varphi)) = \operatorname{Re}\left(\det(Q(\varphi)^*U^*)\prod_{k=1}^n[UQ(\varphi)]_{kk}\right)\\ &= \operatorname{Re}\left(\det(U^*)\prod_{k=1}^nu_{kk}\cdot\frac{1}{u_{ii}u_{jj}}(u_{ii}\cos\varphi+\xi u_{ij}\sin\varphi)(u_{jj}\cos\varphi-\overline{\xi}u_{ji}\sin\varphi)\right)\\ &= -\left(\prod_{k=1}^nu_{kk}\right)\operatorname{Re}\left(\frac{\zeta}{u_{ii}u_{jj}}(u_{ii}\cos\varphi+\xi u_{ij}\sin\varphi)(u_{jj}\cos\varphi-\overline{\xi}u_{ji}\sin\varphi)\right)\\ &= -\left(\prod_{k=1}^nu_{kk}\right)\left(\cos\varphi+\frac{|u_{ij}|}{u_{ii}}\sin\varphi\right)\left(\operatorname{Re}(\zeta)\cos\varphi-\frac{\operatorname{Re}(\zeta\overline{\xi}u_{ji})}{u_{jj}}\sin\varphi\right)\,.\tag{1} \end{align}

Global minimum of function $g$ is obtain whenever the product of the last two factors in $(1)$ is maximized. Using trigonometric addition formulas, we can show this happens for every $\varphi$ satisfying $$2\varphi-\arctan\left(\frac{|u_{ij}|}{u_{ii}}\right)+\arctan\left(\frac{\operatorname{Re}(\zeta\overline{\xi}u_{ji})}{u_{jj}\operatorname{Re}(\zeta)}\right)\in 2\pi\mathbb{Z}\,.\tag{2}$$ On the other hand, global minimum is obtained for $\varphi=0$, because $g(0)=f(U)$. This and $(2)$ implies that $$\operatorname{Re}\big((|u_{ij}|u_{jj}-\overline{\xi}u_{ii}u_{ji})\zeta\big)=0\,.$$ Multiplying the last result with $|u_{ij}|u_{jj}$, we obtain $$\operatorname{Re}\big((|u_{ij}|^2u_{jj}^2-u_{ii}u_{ij}u_{ji}u_{jj})\zeta\big)=0\,.\tag{3}$$

Repeating this procedure with matrix $PUP^T$ obtained from $U$ by exchange of rows $i$ and $j$ and columns $i$ and $j$ gives $$\operatorname{Re}\big((|u_{ji}|^2u_{ii}^2-u_{ii}u_{ij}u_{ji}u_{jj})\zeta\big)=0\,.\tag{4}$$ Repeating the same procedure with matrix $U^*$ gives $$\operatorname{Re}\big((|u_{ji}|^2u_{jj}^2-u_{ii}u_{ij}u_{ji}u_{jj})\overline{\zeta}\big)=0\,.\tag{5}$$

We subtract $(3)$ from $(4)$ to obtain $|u_{ij}|u_{jj}=|u_{ji}|u_{ii}$. From here, it follows that $$u_{ji}=\overline{u_{ij}}u_{jj}u_{ii}^{-1}\rho_{ij}\,,\tag{6}$$ for some $\rho_{ij}$ such that $|\rho_{ij}|=1$.

Using $(6)$ we show that all diagonal elements of $U$ are equal. If necessary, we may symmetrically permute rows and columns of $U$ so that $u_{11}$ is the largest diagonal element. Now, $$1 = \sum_{k=1}^n|u_{1k}|^2 = \sum_{k=1}^n|u_{k1}|^2\frac{|u_{11}|^2}{|u_{kk}|^2} \geq\sum_{k=1}^n|u_{k1}|^2=1\,,$$ from where our claim follows.

Replacing $u_{ji}$ in $(3)$ with $(6)$ implies $u_{ij}=0$ or $\operatorname{Re}((1-\rho_{ij})\zeta)=0$. The same replacement in $(5)$ implies $u_{ij}=0$ or $\operatorname{Re}((1-\rho_{ij})\overline{\zeta})=0$. If $u_{ij}\neq0$, solving this two equations for $\rho_{ij}$ shows that $$\rho_{ij}\in\{1,\zeta^2\}\cap\{1,(\overline{\zeta})^2\}=\{1\}\,.$$ The last equality is true because only solution of $\zeta^2=(\overline{\zeta})^2$ satisfying $\operatorname{Re}(\zeta)>0$ is $\zeta=1$. If $u_{ij}=0$, we are free to take $\rho_{ij}=1$. In any case, $$u_{ji}=\overline{u_{ij}}u_{jj}u_{ii}^{-1}\tag{7}\,.$$ Since all diagonal elements are equal, $(7)$ implies $U$ is a Hermitian matrix.

We now know matrix $U$ is unitary and Hermitian. Therefore, its only eigenvalues are $\pm1$. From $\operatorname{Re}(\zeta)>0$ and $\det(U)\in\{-1,1\}$ we conclude that $\det(U)=-1$, and in turn, that at least one eigenvalue of $U$ is $-1$. Now, $$m=f(U)=-\prod_{k=1}^nu_{kk}=-u_{11}^n = -\left(\frac{1}{n}\operatorname{tr}(U)\right)^n \geq -\left(\frac{n-2}{n}\right)^n\,,$$ shows the conjecture was correct.