How is "point" in geometry undefined? And What is a "mathematical definition"?

I think your question is more about axiomatic systems in general. Maybe this analogy will help: Consider for example the axioms that govern set theory (called "ZFC"). The term "set" there is also undefined - even though we have some intuition about it. From there we then go on to state various properties that sets have to obey.

More generally, when defining an axiomatic system (regardless if it's Euclidean geometry or ZFC set theory), you have "primitive notions" (points or lines resp. sets) and then you state property that relate the various primitive notions to each other.
The main point though is that while we use our intuition to help us find proofs and derive properties, on a formal level these are just manipulations of symbols that are not bound to our intuition. That allows us, if would like to do so, to replace the names of all primitive notions with other names.

Hilbert is famous for making such a remark, where he illustrates this idea taken to the extreme: "One must be able to say at all times--instead of points, straight lines, and planes--tables, chairs, and beer mugs" (source: Provenance of Hilbert quote on table, chair, beer mug , where you can find also a bit of history).

Note that it says on Wikipedia that

[...] in Euclidean geometry, a point is a primitive notion upon which the geometry is built. Being a primitive notion means that a point cannot be defined in terms of previously defined objects. That is, a point is defined only by some properties, called axioms, that it must satisfy. In particular, the geometric points do not have any length, area, volume, or any other dimensional attribute. A common interpretation is that the concept of a point is meant to capture the notion of a unique location in Euclidean space [...]

So the reason that a point would be undefined is because it is a foundation upon which Euclidean Geometry is built. If we were to define a point as a "location that had no size i.e. no width, no length, and no depth", than we would have to define the terms "width", "length" and "depth", and these cannot be defined without using points in some fashion, rendering the definition circular.

As for what counts as a definition, you may want so see here.

Concerning your definition of "point" in geometry, what littleO said in a comment is enough:

You have only replaced one undefined term with other undefined terms.

To be more explicit, if you define "point" in terms of "location", I would simply ask you to define "location".

Now this does not address your other questions.

If it is not a definition, then how can we know whether some statement is definition or not?

Informally, we say that a valid definition is a way of describing something that is precise and only involves previously defined concepts. This is good enough for informal mathematics, but there is in fact a completely precise definition of "valid definitions" in first-order logic, which is variously called definitorial expansion or full abbreviation power. This rule basically allows one to name and later use any constant-symbol or predicate-symbol or function-symbol that can be represented uniquely by some first-order formula. For example, if you work within first-order Peano Arithmetic plus full abbreviation power, you can define:

$even(n) \overset{def}\equiv \exists k\ ( k+k = n )$.

And from then on you can reason about objects that satisfy the now defined predicate $even$, and can prove theorems involving it such as:

$\forall n\ ( even(n \times n) \to even(n) )$.

Suffice to say that such a technical device is necessary in practice so that we do not have pointless duplication of content. Of course the above theorem could have been written in a plain arithmetic sentence without using $even$, but clearly it would be much longer and less informative.

What are the characteristics of a definition in math?

The above concerns the technical details of how to formally define "valid definition" in first-order logic, and hence most of mathematics (which is based on a first-order set theory called ZFC). The concept of abbreviation power extends easily to other logics anyway. But there is a second issue of the different kinds of definitions in mathematics.

The first kind involves defining concepts within an existing framework. The example of $even$ is one instance of this. The second kind involves defining an entire framework (as a single concept)! For instance, we can define a structure to be a model for Peano Arithmetic iff it obeys all the axioms of PA. Note carefully that such a definition does not define what a single natural number is, but what is the collection of natural numbers together with the arithmetic operations as a whole.

Similarly, in any usual axiomatization of Euclidean geometry one does not define what points are, but rather defines that a structure is a Euclidean geometry iff it consists of lines and points (and usually numbers) that together satisfy certain axioms.