What is the state of the equilibrium for a second derivative equal to zero?

Consider the following potentials:

\begin{align} U(x) &= x^4 \\ U(x) &= x^6 - x^4 \\ U(x) &= x^4 + x^3 \end{align}

All three of these potentials have an equilibrium point at $x = 0$. All three of these potentials are such that the second derivative of $U(x)$ at this equilibrium point is zero. However, you should convince yourself (perhaps by plotting these potentials) that in the first case the equilibrium is stable, in the second case it is unstable, and in the third case the equilibrium is, as you put it, "stable in one direction but unstable in the other".

The moral is: knowing only that the second derivative is zero tells us nothing about stability. We need to look at higher derivatives if we want to know more.

First of all you have an imprecise idea about stability: also small velocities matter not only small displacements from the equilibrium. An equilibrium $x_0$ is stable if the motion is confined around $x_0$ and its velocity is confined around the vanishing velocity for every positive time, for every initial condition close to $x_0$ and every initial velocity close to $0$ at time $t=0$.

In other words, according to the general theory of stability (e.g., see Arnold's or Fasano-Marmi's textbooks) the equilibrium $x_0$ is stable (in the future) if

fixing a neighborhood $U$ of $(x_0,0)$, there exists a second neighborhood $V\subset U$ of $(x_0,0)$ such that, every pair of initial conditions $x(0)=y_0$ and $\dot{x}(0) = \dot{y}_0$ with $(y_0, \dot{y}_0) \in V$ gives rise to a motion $x=x(t)$ such that $(x(t), \dot{x}(t)) \in U$ for every $t\in (0, +\infty)$.

A theorem (as above I restrict myself to the one-dimensional case) proves that if all forces are conservative then

(a) a configuration $x_0$ is an equilibrium if and only if $\frac{dU}{dx}|_{x_0}=0$,

(b) an equilibrium $x_0$ is stable if $U$ has a strict minimum at $x_0$ (i.e. $U(x)>U(x_0)$ for $x\neq x_0$ in a neighborhood of $x_0$).

(c) an equilibrium $x_0$ is unstable if $\frac{d^2U}{dx^2}|_{x_0}>0$.

The condition in (b) is satisfied if $\frac{d^2U}{dx^2}|_{x_0} <0$, but this is just a sufficient condition (think of $U(x)=x^4$ with $x_0=0$, it is evidently stable and satisfies (b), but $\frac{d^2U}{dx^2}|_{x_0}=0$).

It remains open the case $\frac{d^2U}{dx^2}|_{x_0}=0$. It has to be studied case-by-case. However certain cases are easy. In particular consider any point $x_0 > x_5$ in your picture. It is clear that the condition in (a) is true, so that $x_0$ is an equilibrium and also $\frac{d^2U}{dx^2}|_{x_0}=0$.

However, perhaps contrarily to the naive idea, $x_0>x_5$ is unstable. Indeed if you start with an initial condition $x(0) = y_0$ arbitrarily close to $x_0$ and a speed $\dot{x}(0) = \dot{y}_0 > 0$ arbitrarily close to $0$, the arising motion is $x(t) = \dot{y}_0 t + y_0$ and, waiting a sufficiently large time $t>0$, $x(t)$ exits from every neighborhood of $x_0$ initially fixed.

(Statement (b) is nowadays an elementary subcase of a famous theorem due to Lyapunov but a proof was already known by Lagrange and Dirichlet. As a matter of fact, the total energy $E(x, \dot{x})$ is a Lyapunov function for the system for the critical point $(x_0,0)$ when $U$ has a strict minimum at $x_0$.)

Taylor expand the force $F(x) = -U'(x)$ about $x = x_0$:

$$F(x_0 + \Delta x) = -U'(x_0) - U''(x_0)\Delta x - \frac{1}{2}U'''(x_0)(\Delta x)^2 + \cdots$$

Stipulate that $F(x_0) = 0$ and then

$$F(x_0 + \Delta x) = -U''(x_0)\Delta x - \frac{1}{2}U'''(x_0)(\Delta x)^2 + \cdots$$

In the case that $U''(x_0) \gt 0$, then for $\Delta x$ small, the force is approximately a linear restoring force.

However, in the case that $U''(x_0) = 0$ (and at least one higher order derivative is non-zero), then for $\Delta x$ small, the force is non-linear and not necessarily a restoring force.

For example, if $U'''(x_0) \ne 0$, then the sign of the force does not change as $\Delta x$ goes through zero; the force is opposite the displacement in one direction and with the displacement in the other direction (which will drive the particle away from $x=x_0$).

For the force to be restoring in the case that $U''(x_0) = 0$ requires that the lowest order non-zero derivative (higher than the 2nd) be even order and positive