What is a point?

Point, in Euclidean geometry, is an undefined notion.

We do not define what a point is, only what properties points must have, and these properties are completely specified by the axioms. This is certainly a modern view of mathematics, and differs from the approach in Euclid's times. Euclid defines point, but the definition is vague, and it is never used anyway.

The modern view can be seen in Hilbert's book "Foundations of geometry", where a modern treatment of geometry is given, with emphasis on the axiomatic approach. To see the extent to which we do not care about what a point could possibly be, see this question, on a famous quote by Hilbert stating that "One must be able to say at all times -- instead of points, straight lines, and planes -- tables, chairs, and beer mugs".

Of course, we usually want to work with very concrete "models" of the axioms, the most famous being the Cartesian plane. In this model, the plane is just $\mathbb R\times\mathbb R$, and we identify a point with an element of this set, that is, an ordered pair $(x,y)$.

Also, there are several (equivalent) ways of axiomatizing geometry, so details of what "basic properties" we assume as a priori will vary depending on what concrete axiomatization or specific model of the axioms one has in mind or is working with.

We can't always define everything or prove all facts. When we define something we are describing it according to other well-known objects, so if we don't accept anything as obvious things, we can not define anything too! This is same for proving arguments and facts, if we don't accept somethings as Axioms like "ZFC" axioms or some else, then we can't speak about proving other facts. About your question, I should say that you want to define "point" according to which objects? If you don't get it obvious you should find other objects you know that can describe "point"!