Completeness in Krasner's Lemma

^{I would put this "answer" into comment since I only give references. Unfortunately I don't have enough reputation to do so.}

Your thought is correct, for s proof see e.g. Theorem 19 in this note.

Also in Neukirch's [ANT, Exercise II.6.2] about Krasner's Lemma the henselian assumption suffices, though it isn't written explicitly there.

For characteristic $0$ the proof for Henselian from complete is easy.

$K$ is Henselian (for $v$) means that $K = K_v\cap \overline{K}$ where $v$ is a non-trivial discrete valuation on $K$ and $K_v$ is the completion for $|.|_v$ and $\overline{K}$ is the algebraic closure.

For $L/K$ a finite extension and $F$ its Galois closure and $\sigma \in Hom_K(L,F)$, extend $v$ to $F$, then $\sigma$ is a $K$-linear map $L\to F$, being finite dimensional $K$-vector spaces $\sigma$ is continuous for $|.|_v$. Thus it extends to the completion $$\sigma\in Hom_{K_v}(L_v,F_v)$$ where $v|_{F_v}$ has to be the unique extension of $v|_{K_v}$.

Consequently $$[L:K] = |Hom_K(L,F)|=|Hom_{K_v}(L_v,F_v)|=[L_v:K_v]$$ and $$[L:K]=[L_v:K_v]=[(L_v\cap \overline{K})_v:K_v]=[L_v\cap \overline{K}:K]$$ ie. $L=L_v\cap \overline{K}$ is Henselian.

$L$ is fully determined by its uniformizer and residue field. Krasner lemma for complete fields implies that $L\to L_v, L_v\to L_v\cap \overline{K}$ is a bijection between the finite extensions of $K$ and the finite extensions of $K_v$. Thus Krasner lemma applies to both.