What holds up the lowest point of a rope following a catenary curve?

That lowest point is not exactly horizontally aligned with the neighboring points. It is slightly below.

If it wasn't, then, as you describe, there would be a net vertical force downwards. If that was the case, then the point would accelerate downwards until that vertical force was balanced. And then you would again reach the configuration I mentioned: the bottom point is not perfectly horizontally aligned with it's neighbors; it is slightly lower.

If you now would argue that we can reduce the size of that point towards being infinitesimal causing it's position to tend towards horizontal alignment, then you are forgetting that the mass also at the same time tends towards being infinitesimal; towards zero, basically. Having a mass of zero means no gravitational force downwards and thus no vertical net force anyways.

Such an ultimately lowest and infinitesimally small point does not need to be held up, because it isn't falling.

The rope is continuous and of uniform density. The amount of mass any small segment has depends on how long this segment is.

At exactly the center you either have a segment of zero length and hence no mass and weight, or a segment of finite length, but with its ends curved up every so slightly providing just enough vertical force to support the weight.

That is what a catenary is by definition.