What is the exact value of $\eta(6i)$?

After persevering with a Mathematica session, I found that $F(6i)$ is the root of $96$-deg eqn (no wonder it was hard to find!) but could be prettified as,

$$\eta(6i) = \frac{1}{2\cdot 6^{3/8}} \left(\frac{5-\sqrt{3}}{2}-\frac{3^{3/4}}{\sqrt{2}}\right)^{1/6}\,\frac{\Gamma\big(\tfrac{1}{4}\big)}{\pi^{3/4}}$$

However, the second question is still open.

Since we know the value of $\eta(3i)$, the point is just to compute the value of the product: $$ \prod_{n\geq 0}(1+e^{-6\pi n})=\exp\sum_{n\geq 0}\log\left(1+e^{-6\pi n}\right)=\exp\sum_{n\geq 0}\int_{n}^{n+1}\frac{6\pi n}{1+e^{6\pi s}}\,ds$$ where: $$\sum_{n\geq 0}\int_{n}^{n+1}\frac{6\pi n}{1+e^{6\pi s}}\,ds = \int_{0}^{+\infty}\frac{6\pi s\,}{1+e^{6\pi s}}-\int_{0}^{+\infty}\frac{6\pi\{s\}}{1+e^{6\pi s}}$$ and the first integral in the RHS equals $\frac{\pi}{72}$ by the residue theorem, while expanding the fractional part as its Fourier series, $\{s\}=\frac{1}{2}-\sum_{n\geq 1}\frac{\sin(2\pi n s)}{\pi n}$, we get:

$$\begin{eqnarray*}\int_{0}^{+\infty}\frac{6\pi\{s\}\,ds}{1+e^{6\pi s}}&=&\frac{\log 2}{2}-\sum_{n\geq 1}\int_{0}^{+\infty}\frac{6 \sin(2\pi n s)}{n(e^{6\pi s}+1)}\,ds\\&=&\frac{\log 2}{2}-\sum_{n\geq 1}\frac{6}{n}\sum_{m\geq 0}(-1)^m\int_{0}^{+\infty}\sin(2\pi n s)\,e^{-6\pi m s}\,ds\\&=&\frac{\log 2}{2}-\frac{3}{\pi}\sum_{n\geq 1}\sum_{m\geq 0}\frac{(-1)^m}{9 m^2+n^2}\\&=&\frac{\log 2}{2}-\frac{\pi}{2}-\frac{3}{\pi}\sum_{m,n\geq 1}\frac{(-1)^m}{9m^2+n^2}\end{eqnarray*}$$ and the last series just depends on the number of ways to represent a positive integer $\not\equiv 2\pmod{3}$ through the binary quadratic form $n^2+9m^2$: it is, with minor manipulations, just a Dirichlet convolution. I have just applied the same techniques of this answer, just in reverse.

This shows a clear connection between the evaluation of the Dedekind eta function at quadratic irrationals and the class number problem: $\eta(\sqrt{-N})$ depends on $\sum_{n\geq 1}(-1)^n\frac{r(n)}{n}$, where $r(n)$ counts the number of ways to represent $n$ as $a^2+Nb^2$. If $N$ is a square or $a^2+Nb^2$ is the only reduced quadratic form of discriminant $-4N$ (class number one) we may explicitly compute such series, and it turns out that $F(\sqrt{-N})$ is an algebraic number. Otherwise, $\sum_{n\geq 1}(-1)^n\frac{r(n)}{n}$ is not even a convolution of Dirichlet series, hence your conjecture is very likely to hold.

Ultimately, the computation of $\eta(6i)$ can be carried on by recalling that:

$$j(\tau)=\left(\left(\frac{\eta(\tau)}{\eta(2\tau)}\right)^8+2^8\left(\frac{\eta(2\tau)}{\eta(\tau)}\right)^{16}\right)^{3} $$ and by computing the Klein $j$-invariant $j(3i)$. The Wikipedia page gives:

$$ j(3i) = \frac{1}{27}(2+\sqrt{3})^2(21+20\sqrt{3})^3.$$

The answer to the second question is very simple. Assume the complete elliptic integral of the first kind $$ K(x)=\int^{\pi/2}_{0}\frac{d\theta}{\sqrt{1-x^2\sin^2(\theta)}}=\frac{\pi}{2}\cdot {}_2F_1\left(\frac{1}{2},\frac{1}{2};1;x^2\right). $$ Then if we define the elliptic singular modulus $0<k_r<1$ to be the solution of $$ \frac{K\left(\sqrt{1-k_r^2}\right)}{K(k_r)}=\sqrt{r}\textrm{, }r>0, $$ we know that $k_r$ is algebraic for $r$ positive rational. Assume the notation: $$ K[r]:=K(k_r)\textrm{, }r>0 $$ Also for $r_1,r,N$, such that $N$ positive integer, $r\in Q^{*}_{+}$, with $r_1=N^2r$, we have $$ \frac{K[N^2r]}{K[r]}=M_N(r) $$ The function $M_N(r)$ is called: "the multiplier" and for $r,N$ as above it is an algebraic number.

The Dedekind eta function is just $$ \eta(z)=\eta_1(q):=q^{1/24}\prod^{\infty}_{n=1}\left(1-q^n\right)\textrm{, }q=e^{2\pi i z}\textrm{, }Im(z)>0.\tag 1 $$ We want to find $\eta(i\sqrt{r})$, $r>0$. For this we define the Ramanujan eta function $$ f(-q):=\prod^{\infty}_{n=1}\left(1-q^n\right)\textrm{, }q=e^{-\pi\sqrt{r}}\textrm{, }r>0.\tag 2 $$ The function $f(-q)$ can evaluated at $q=e^{-\pi\sqrt{r}}$, $r>0$ as (see [W,W] Chapter 21 pg.488): $$ f(-q)=\frac{2^{1/3}}{\sqrt{\pi}}q^{-1/24}(k_r)^{1/12}(k'_r)^{1/3}K[r]^{1/2}\tag 3 $$ Assume that $q_1=e^{-\pi\sqrt{r_1}}$, with $r_1=N^2r$ and $GCD(N,r)=1$, ($r_1,N,r$ are integers, $N>1$ and $r$ is square free), then from (1),(2),(3) we have: $$ \eta(i\sqrt{r_1})=\eta(i\sqrt{N^2r})=q_1^{1/12}f(-q_1^2)=algebraic\cdot \frac{\sqrt{K[4r_1]}}{\sqrt{\pi}}=alg\cdot\frac{\sqrt{K[4N^2r]}}{\sqrt{\pi}}= $$ $$ =alg\cdot \sqrt{M_2(r)}\sqrt{M_{N}(r)}\frac{\sqrt{K[r]}}{\sqrt{\pi}} $$ Hence if $r_1$ is a square (which means $r=1)$, then $$ \eta(i\sqrt{r_1})=alg\cdot\sqrt{\frac{K[1]}{\pi}}=alg\cdot \frac{\Gamma\left(\frac{1}{4}\right)}{\pi^{3/4}} $$ Q.E.D.

Note: $k'_r=\sqrt{1-k_r^2}$. $k'_r$ is called the coplementary modulus.

References

[W,W]: E.T. Whittaker and G.N. Watson. 'A course on Modern Analysis'. Cambridge U.P. 1927.