Is the normalizer of a reductive subgroup reductive?
I assume $H$ and $G$ are connected, and will explain a purely algebraic proof of the affirmative answer in characteristic 0 (bypassing Cartan involutions) and address the possibility of failure in positive characteristic by putting the question into a broader context that also addresses Jim Humphreys' question of why this is a natural question to wonder about (though I don't know the OP's motivation).
For reasons that will become clearer below, my expectation is that the answer is negative in positive characteristic, due to the failure of nontrivial connected semisimple groups (unlike tori) to have completely reducible representation theory in such characteristics. (EDIT: McNinch has now given such examples over any algebraically closed field of characteristic $p > 0$: the adjoint semisimple subgroup $H = {\rm{PGL}}_n$ inside ${\rm{SL}}({\mathfrak{gl}}_n)$ embedded via "conjugation" for any $n$ divisible by $p$.) No doubt in char. 0 the argument via Cartan involutions by Aakumadula is simpler than what is below in char. 0 (though the proof of Mostow's theorem upon which Aakumadula's argument rests is not easy).
Let $N_G(H)$ and $Z_G(H)$ denote the respective scheme-theoretic normalizer and centralizer of $H$ in $G$, so we want to determine if $N_G(H)^0_{\rm{red}}$ is reductive. The first point is that (in any characteristic) $N_G(H)_{\rm{red}}^0$ is reductive if and only if $Z_G(H)_{\rm{red}}^0$ is reductive. The "only if" direction follows from the normality of $Z_G(H)_{\rm{red}}$ in $N_G(H)_{\rm{red}}$. For the converse, consider the action of $N_G(H)$ on $H$ through conjugation in $G$. This defines a map from $N_G(H)$ to the automorphism functor of $H$ (over $k$), and since $H$ is connected reductive this functor is represented by a smooth $k$-group scheme ${\rm{Aut}}_{H/k}$ whose identity component is the adjoint quotient $H^{\rm{ad}}$ (by SGA3 technology which could be avoided for our purposes by using some auxiliary ad hoc constructions). In particular, the action of $N_G(H)^0$ on $H$ is given by composing a $k$-homomorphism $$f:N_G(H)^0 \rightarrow {\rm{Aut}}_{H/k}^0 = H^{\rm{ad}}$$ with the natural action of $H^{\rm{ad}}$ on $H$. But $f|_H$ is the natural quotient map (due to how $H^{\rm{ad}}$ is identified with a subgroup of ${\rm{Aut}}_{H/k}$) and $(\ker f)_{\rm{red}}^0 = Z_G(H)^0_{\rm{red}}$, so the insensitivity of reductivity to passing to an isogenous quotient and the reductivity of $H^{\rm{ad}}$ imply that $N_G(H)^0_{\rm{red}}$ is reductive if (and only if) $Z_G(H)^0_{\rm{red}}$ is reductive.
So now we are faced with the question of determining when $Z_G(H)^0_{\rm{red}}$ is reductive. Observe that if $H$ were not given as a $k$-subgroup of $G$ (equipped with its conjugation action on $G$) but rather were given to us as an abstract connected reductive group equipped with an abstract action on $G$ as a $k$-group then we could always form the associated connected reductive semidirect product $G' = G \rtimes H$ in which $H$ lies as a subgroup and for which $$Z_{G'}(H) = G^H \times H$$ (where $G^H$ is the scheme-theoretic centralizer in $G$ of the $H$-action). Thus, the reductivity of $(G^H)^0_{\rm{red}}$ is equivalent to that of $Z_{G'}(H)^0_{\rm{red}}$.
To summarize, the original general problem over $k$ of reductivity of the identity component of normalizers of connected reductive subgroups of connected reductive groups is equivalent to the general problem over $k$ of when the underlying reduced scheme of the centralizer $G^H$ for an abstract action of a connected reductive $k$-group $H$ on a connected reductive $k$-group $G$ has reductive identity component. (In particular, the answer is affirmative to one of these two general questions over $k$ if and only if the answer is affirmative to the other general question over $k$.)
We shall now record a purely algebraic result (valid over algebraically closed fields of any characteristic) that gives an affirmative answer to the above questions in characteristic 0 and illuminates the search for counterexamples in positive characteristic. Recall that an affine $k$-group scheme of finite type is called linearly reductive if its finite-dimensional linear representation theory over $k$ is completely reducible. In characteristic 0 this is equivalent to reductivity of the identity component, whereas in positive characteristic the only linearly reductive $k$-group schemes are the affine finite type $k$-groups whose identity component is multiplicative type and component group has order not divisible by char($k$) (by a theorem of Nagata).
$\mathbf{Theorem}$: If $H$ is a linearly reductive $k$-group equipped with an action on a connected reductive $k$-group $G$ then $(G^H)^0_{\rm{red}}$ is reductive.
This (hard) theorem is Proposition A.8.12 of the book "Pseudo-reductive groups". (When ${\rm{char}}(k) = p > 0$ this Theorem is useful for $H = \mu_{p^n}$, not just for the rather classical case when $H$ is a torus.) The proof rests on a beautiful result due independently to Borel (using etale cohomology) and Richardson (using Haboush's theorem), and it also involves considerations with non-representable functors.
Observe that finding a counterexample to the original problem in positive characteristic amounts to showing that the above Theorem has optimal hypotheses; e.g., it fails for some connected semisimple $H$ (and a suitable $G$). This is a very natural question in positive characteristic, and so (as I hope Jim Humphreys will agree) adequately motivates the original problem.
The answer is yes, at least in characteristic zero. There is a Theorem of Mostow which says that $G$ may be viewed as a subgroup of $GL_n$ such that the restriction of the Cartan involution of $GL_n({\mathbb C})$ to $G$ and $H$ gives Cartan involutions on $G$ and $H$, Therefore, the normaliser $N_G(H)$ of $H$ in $G$ is also invariant under this Cartan involution. Hence it is reductive.
In positive characteristic, the normalizer of a (connected) reductive subgroup of a (connected) reductive group is not in general reductive.
I communicated the example below in an emailed answer to a query in April 2002 (it took me some searching in old emails to find it!) At the time I wrote something at the end like "I'm not aware of a good reference" -- this remains true.
Let $k$ be alg. closed of positive characteristic. I will exhibit $H \subset G$ reductive groups such that $C_G(H)$ is not reductive; since $C_G(H)$ is normal in $N_G(H)$, also $N_G(H)$ is not reductive.
Here is the example. Let $n \ge 2$, and consider the adjoint representation $$\operatorname{Ad}: \operatorname{GL}_n \to \operatorname{GL}(\mathfrak{gl}_n)$$ where $\mathfrak{gl}_n$ is the Lie algebra of $\operatorname{GL}_n$.
The image $H \simeq \operatorname{PGL}_n$ of $\operatorname{Ad}$ is a reductive subgroup of $G=\operatorname{GL}(\mathfrak{gl}_n)$, and the centralizer $C_G(H)$ identifies with the group of units of the endomorphism ring $\operatorname{End}_H(\mathfrak{gl}_n)$.
If $(n,p) = 1$, $\mathfrak{gl}_n$ is a semisimple representation of $\operatorname{GL}_n$ with two distinct irreducible factors, so that $\operatorname{End}_H(L) = k \times k$. In this case $C_G(H)$ is a 2 dimensional torus and hence is reductive.
If $(n,p) = p$, $\mathfrak{gl}_n$ is an indecomposable representation with 3 composition factors. Thus the endomorphism ring of $\mathfrak{gl}_n$ is a local ring. It turns out that this endomorphism ring $\operatorname{End}_H(\mathfrak{gl}_n)$ still has dimension 2, but it is now isomorphic to the ring $k[t]/\langle t^2 \rangle$. To exhibit a non-0 nilpotent $H$-endomorphism $t$ of $\mathfrak{gl}_n$, take the following map: to a matrix $X$ in $\mathfrak{gl}_n$, assign the matrix $t(X) = \operatorname{tr}(X).1$ where 1 denotes the $n\times n$ identity matrix, and $\operatorname{tr}()$ denotes the trace. Since $n$ is divisible by $p$, applying $t$ twice gives 0.
The unit group of this local ring is isomorphic to the product of a $1$ dimensional torus and the additive group of the field; such a group is not reductive. (Explicitly: the additive subgroup is precisely the set of all automorphisms $1 + at$ of $\mathfrak{gl}_n$ with $a \in k$.)
More generally, let $H$ be any reductive group, let $V$ be a finite dimensional $H$-module, and write $G = \operatorname{GL}(V)$. Suppose that $V$ is indecomposable, has composition length 3, and that $\operatorname{soc}(V) \simeq V /\operatorname{rad}(V)$. Then $\operatorname{End}_H(V) \simeq k[t] / \langle t^2 \rangle$ and $C_G(H) = \operatorname{End}_H(V)^\times$ is not reductive.
Here are a few more examples of such $H$ and $V$ (I'll write $T(\mu)$ for the indecomposable tilting module with highest weight $\mu$).
$H = \operatorname{Sp}(W)$, $V = \bigwedge^2 W$ ("exterior square"), when $p>2$ and $\dim W \equiv 0 \pmod p$.
$H = \operatorname{SO}(W)$, $V = \operatorname{Sym}^2 W$ ("symmetric square"), when $p>2$ and $\dim W \equiv 0 \pmod p$.
$H = \operatorname{SL}_2$, $V = T(n)$ for $p \le n \le 2p-2$.
$H = \operatorname{SL}_{\ell + 1}$, $V = T(\varpi_i + \varpi_\ell)$ for $1 \le i < \ell$, when $\ell + 2 - i \equiv 0 \pmod p$, [This more-or-less interpolates the original example since when $i=1$ and $\ell + 1 \equiv 0 \pmod p$, $T(\varpi_1 + \varpi_\ell) \simeq \mathfrak{gl}_{\ell+1}$.]
$H = \operatorname{SO}_{2\ell}$, $V = T(\varpi_1 + \varpi_\ell)$ when $\ell \equiv 0 \pmod{p}$.