Doesn't the work $W = \int F \, dx$ count only the work done by the outermost point of a spring?

Suppose that there is a spring fixed at one end and a force $F$ is applied to the spring.
The important thing is that force $F$ can be thought of as the "tension" in the spring and is the same throughout the spring.

Now consider the whole spring as three equal lengths.

Each length is subjected to a force $F$ at each end as shown in the diagram.

The work done on spring 1 by the forces stretching it by a small amount $\delta x$ is $F\,\delta x$.
The work done on spring 2 by the forces stretching it by $\delta x$ is $F\,2\delta x - F\,\delta x = F\,\delta x$.
The work done on spring 3 by the forces stretching it by $\delta x$ is $F\,3\delta x - F\,2\delta x = F\,\delta x$.

So the total work done in stretching the three springs is $F\,\delta x +F\,\delta x+F\,\delta x = F\,3\delta x = F\Delta x$ where $\Delta x (=3\delta x)$ is the total extension of the whole spring.

The whole spring can be cut up into as many segments as you like and then the work done extending the spring can be written as $\displaystyle \int_0^{x_{\rm final}} F\,dx$ where $x_{\rm final}$ is the final extension of the whole spring.

When we are integrating $(-kx)dx$, we are calculating the work done on displacement $dx$ off the point which is in the natural length position, (i.e. the point we begin applying force in) with force $kx$ ($x$ varies from $0$ to $x$ for each $dx$). This means we are calculating the total work by the spring force of the outermost point only. But all the points in the spring are displaced. Why don't we add all the work done by them?

The formula $\int(-kx)dx$ has within it the spring constant $k$, which has units of N/m. This is, of course, a linear approximation for small displacements. The infinitesimal displacement $dx$, multiplied by the spring constant, will result in an infinitesimal amount of Work, and integration/summing of all those infinitesimals will yield a total amount of work being done over the total displacement from $0$ to $x$.

In your question, it appears you have assumed that because only the position of the end of the spring is being considered in the integration, that the rest of the spring is being ignored/not considered by the integration?

But, in fact, the spring constant $k$ includes/implicitly incorporates the force contributed by every increment of the entire spring. Every increment of the spring must be stretched with every displacement of the spring's end.

Obviously, the spring constant $k$ will not be as large if the spring consisted of only an infinitesimally small increment of spring at the end of the spring. In other words, the spring constant has the value $k$ because of the total contribution of all the increments of force that must be overcome with each $dx$ of displacement of the end of the spring.

In other words, the concept/actuality of the spring constant $k$ includes the force applied against displacement of all the increments of the spring.

Thus, when integrating the force multiplied times each increment of displacement, the integration naturally/unavoidably includes the total force applied to displace all the spring's constituent inter-atomic bonds. Thus, the Work calculated for each increment of spring-end displacement implicitly includes the force needed to displace all of the interatomic bonds in the entire spring. And subsequently, integration of all those increments of Work yields the total work done by displacing the spring against the force exerted by all the bonds within the spring.

Why don't we sum (integrate) the work done by all the points?

In essence we are doing just that. When we take the integral of $(kx)dx$ between zero (the relaxed position of the end of the spring) to $x$, the stretched position, we are macroscopically summing up the work done on the microscopic level to pull the molecules apart at each point along the length of the spring in opposition to the electrostatic forces between them.

Envision the spring being comprised of molecules connected together by microscopic springs between them. The spring constant, $k$, is the macroscopic parameter equivalent to the microscopic spring constant. The integral of $(kx)dx$ sums up all the work being done on the microscopic level in stretching the microscopic springs.

But then are summing work done on each point only for distance dx

What I mean is the sum of all work at the molecular level equals the total work at the macroscopic level, and the sum of all the intermolecular displacements, $dx_i$, add up to the final displacement, $x$ as @Farcher showed pictorially. Or

$$\sum_{i=0}^{n}k_{i}dx_{i}=\frac{kx^2}{2}$$

Where $\sum dx_{i}$ = $x$.

Hope this helps.