What is the exact difficulty in defining a point in Euclidean geometry?

The exact difficulty is that in mathematics, we define things in terms of other things. We also avoid circular definitions, in other words we do not want to define $A$ in terms of $B$ where $B$ is defined in terms of $C$ and $C$ is defined in terms of $A$.

Because we avoid circularity, we can lay out all our definitions in order so that anything that is defined is defined when it is first mentioned. But if we write out a mathematical theory in this fashion, and look at the very first definition we wrote, the thing it defined was defined in terms of some other things, and those "other things" cannot have been defined previously (since this is our first definition) and will not be defined later (since every definition comes before the first use of the thing it defines).

In short, in order to build a mathematical theory we have to start with some "primitive notions" that we will never define. Everything else can be defined in terms of those notions.

This does not prove that a point must be one of the primitive notions of Euclidean geometry, but it happens that it has been chosen to be one of those primitive notions, and this choice has worked out well.

Under your definition, the following things are points:

• the variable $x$
• the sentence $(\forall x)(\exists y)(x^2 = y)$
• $\int_{\infty}^{\infty} f(x) dx$ where $f$ is unspecified
• the category of all small categories

I'm sure you agree that none of these things are points.

In a famous comment in the context of a discussion of axiomatisations, David Hilbert noted that these undefined objects might as well be beermugs. The basic conceptual problem here is that mathematicians are tempted to look for ultimate foundations for mathematics. This naturally leads one to a stalemate because whatever foundation you declare to be ultimate, will contain undefined terms which you can in turn ask for even more rigorous foundations for.

The solution is to abandon foundationalism altogether together with a quixotic search for ultimate rigor, and view axiomatisations for what they are, namely convenient tools for clarifying the relations among mathematical entities one is interested in. And of course I will refuse to define a mathematical entity just I would refuse foundationalism.