Can someone explain the math behind tessellation?

What's going on is a little group theory, specifically the study of wallpaper groups. The best introduction I know to this subject (that specifically cares about tessellations and is written for non-mathematicians) is Conway, Burgiel, and Goodman-Strauss's The Symmetries of Things. There are diagrams in the latter book which should be quite helpful.

As far as calculating positions, I guess for starters you're asking for the coordinates of the points on a hexagonal lattice. These are exactly the points

$$\left( n + \frac{m}{2}, \frac{ \sqrt{3}}{2} m \right)$$

where $m, n$ are integers. This is a fairly straightforward calculation using the side lengths of a 30-60-90 triangle (half of an equilateral triangle), which can be deduced, for example, using the Pythagorean theorem.

In addition to some of the "technical" issues involved in making a particular design (treated above) there is also the issue of the difference between symmetry of a pattern when the colors are disregarded and the symmetry of the pattern that includes the colors. In addition to the book: Symmetry of Things, which does treat color symmetry, another book that treats color symmetry is Tilings and Patterns by Branko Grünbaum and Geoffrey Shephard (W.H. Freeman), 1987.

I recommend reading "Symmetry" by Hermann Weyl. He first discusses the history of symmetry and then classifies the point symmetry groups, the frieze groups and finally the wallpaper groups. Note that spherical tilings are just as interesting (and as storied) as Euclidean tilings. Hyperbolic tilings are more recent -- they are a less fundamental yet somewhat deeper subject.