Smooth function proof
I will try to not only give you a solution but also motivate it a bit:
a) Smoothness of $f$: To show that $f$ is smooth, we have to show that all derivatives $f^{(n)}$ exist. You have already found out that $f'(x) = \frac{-2}{x^3} e^{-1/x^2}$. For $x > 0$, we then get $$f''(x) = \frac6{x^4} e^{-1/x^2} + \frac{-2}{x^3} \cdot \frac{-2}{x^3} e^{-1/x^2} = \frac{6x^2 + 4}{x^6} e^{-1/x^2}.$$ So we have, for $x > 0$, \begin{align*} f(x) &= r_0(x) \cdot e^{-1/x^2} \quad \text{with $r_0(x) = 1$},\\ f'(x) &= r_1(x) \cdot e^{-1/x^2} \quad \text{with $r_1(x) = \frac{-2}{x^3}$},\\ f''(x) &= r_2(x) \cdot e^{-1/x^2} \quad \text{with $r_2(x) = \frac{6x^2 + 4}{x^6}$}. \end{align*} The $r_i$ are rational functions (quotients of polynomials) and it is easy to see that the derivative of a rational function is again a rational function whose denominator has no additional zeros.
In the above equations there clearly is a pattern. So let us conjecture the following: For all $n \ge 0$ and all $x > 0$, $f^{(n)}(x)$ exists and has the form $f{(n)}(x) = r_n(x) \cdot e^{-1/x^2}$ where $r_n$ is a rational function whose denominator is some power of $x$.
Lets prove that by induction on $n$: The base case $n = 0$ is clear. Now assume that for some $n$ the assertion is correct. Then we get for all $x > 0$: \begin{align*} f^{(n+1)}(x) = (f^{(n)})'(x) = \left(r_n(x) \cdot e^{-1/x^2}\right)' = r_n'(x) \cdot e^{-1/x^2} + r_n \cdot \frac{-2}{x^3} e^{-1/x^2} = \left(r_n'(x) - \frac{2 r_n(x)}{x^3}\right) e^{-1/x^2}. \end{align*} Define $r_{n+1}(x)$ to be the left factor in this term. Then it is not too hard to check that $r_{n+1}$ is a rational function whose denominator is a power of $x$. In the following, let us write $r_n(x) = p_n(x) / x^{k_n}$ for some polynomial $p_n$.
So far we have shown that $f$ is smooth for every $x > 0$. Also, it is trivial that $f$ is smooth for $x < 0$ and that all derivatives are zero there. Thus it remains to be shown that $f$ is also smooth at $0$. So let us conjecture that $f^{(n)}(0)$ exists and is equal to $0$. This is true for $n = 0$. Now suppose it is true for some $n$. Then $$f^{(n+1)}(0) = \lim_{h \to 0} \frac{f^{(n)}(h)}{h} = \lim_{h \to 0} \frac{r_n(h)}{h \cdot e^{1/x^2}} = \lim_{h \to 0} \frac{p_n(h)}{h^{1+k_n} \cdot e^{1/x^2}} = 0.$$ The last step is l'Hospital's rule (applied multiple times).
Hence $f$ is smooth.
The properties of $g$ are clear. The properties of $h$ are also not too hard to see. Not that the infinite integral $\int_{\infty}^{\infty} g(x) dx$ is actually finite, since $g$ is zero outside of $(a, b)$.
b) Lets first consider the case $k = 1$. Take $F = 1 - h$ on [a, b]. Then $F$ is a smooth function with $F(x) = 1$ for $x = a$, $1 > F(x) > 0$ for $x \in (a, b)$ and $F(x) = 0$ for $x = b$. Thus $F$ has the desired properties for $x \in [a, b]$. We extend $F$ by saying $F(x) = 1$ for $|x| < a$, $F(x) = 0$ for $|x| > b$ and $F(x) = F(|x|)$ for $x \in [-b, -a]$.
Check that this $F$ is smooth. Then certainly $F$ is the function we are looking for.
Now, for a general $k$, we can try a similar thing. On $\Bbb R^k$ we have the norm $\|x\|$ which is smooth for $x \ne 0$. Using this norm and the function $F$, can you find a solution for b)?