Behavior of a sum on the boundary of convergence/divergence

We have $\mathcal{L}(\log x)=-\frac{\gamma+\log(s)}{s}$, hence $\mathcal{L}^{-1}\left(\frac{\log s}{s}\right) =-\left(\log(x)+\gamma\right)$. Additionally $$ \mathcal{L}\left(\frac{\log x}{x}\right) = \frac{\pi^2}{12}+\frac{\left(\gamma+\log x\right)^2}{2},$$ hence a pretty good approximation is $$ \sum_{n\geq 1}\frac{\log n}{n}\,x^n \sim \frac{(\gamma+\log(1-x))^2}{2}\quad \text{for }x\to 1^- $$ by just rephrasing Antonio Vargas' approach in terms of $\mathcal{L}/\mathcal{L}^{-1}$ and Hardy Littlewood's tauberian theorem.

Here are some notes [PDF] on this topic. The basic idea is to compare the sum to the corresponding integral, which is often easier to estimate.

Lemma 3 of those notes:

For a given function $\psi : [N,\infty) \to \mathbb R^+$ suppose that the map $t \mapsto \psi(t) x^t$ is unimodal with maximum at $t = t_x \geq N$, $0 < x < 1$. Then $$ \sum_{n \geq N} \psi(n) x^n = \int_N^\infty \psi(t) x^t\,dt + O\left(\psi(t_x)x^{t_x}\right) + O(1) $$ as $x \to 1^-$.

The map $t \mapsto \frac{\log t}{t} x^t$ is indeed unimodal, and $\frac{\log t}{t} x^t \leq 1/e$ for all $0 \leq x \leq 1$. Thus

$$ \sum_{n=1}^{\infty} \frac{\log n}{n} x^n = \int_1^\infty \frac{\log t}{t} x^t\,dt + O(1) \tag{$*$} $$

as $x \to 1^-$. If we set $\lambda = -1/\log x$ (so that $\lambda \to \infty$ as $x \to 1^-$) and make the substitution $s = t/\lambda$, the integral becomes

$$ \begin{align} \int_1^\infty \frac{\log t}{t} x^t\,dt &= \int_1^\infty \frac{\log t}{t} e^{-t/\lambda}\,dt \\ &= \log \lambda \int_{1/\lambda}^{\infty} \frac{e^{-s}}{s}\,ds + \int_{1/\lambda}^{\infty} \frac{e^{-s} \log s}{s}\,ds. \tag{$**$} \end{align} $$

Skipping some of the details,

$$ \int_{1/\lambda}^{\infty} \frac{e^{-s}}{s}\,ds \sim \int_{1/\lambda}^{1} \frac{e^{-s}}{s}\,ds \sim \int_{1/\lambda}^{1} \frac{1}{s}\,ds = \log \lambda $$

and

$$ \int_{1/\lambda}^{\infty} \frac{e^{-s} \log s}{s}\,ds \sim \int_{1/\lambda}^{1} \frac{e^{-s} \log s}{s}\,ds \sim \int_{1/\lambda}^{1} \frac{\log s}{s}\,ds = - \frac{(\log \lambda)^2}{2} $$

as $\lambda \to \infty$.

By substituting these into $(**)$ we get

$$ \int_1^\infty \frac{\log t}{t} x^t\,dt \sim \frac{(\log \lambda)^2}{2} = \frac{(-\log(-\log x))^2}{2} \sim \frac{(\log(1-x))^2}{2} $$

and hence

$$ \sum_{n=1}^{\infty} \frac{\log n}{n} x^n \sim \frac{(\log(1-x))^2}{2} $$

as $x \to 1^-$ by $(*)$.