What's the point of a Horseshoe map?
The horseshoe map is one example of a discrete-time dynamical system with interesting properties. Consider what happens with two points $x_1$ an $x_2$ which are arbitrarily close together and track their images under iterated application of the horseshoe map. Are their orbits (i.e. the sets $\{f^n(x_i)\}_{n=1,2,\dots}$ related? Do they stay close together? In fact they don't. The dynamics of this map is chaotic, consider the Wikipedia page for further clarification.
By the way: Have you ever tried to mix two colors of colored putty? To simply "knead" randomly will not do very good. However, something "horseshoeisch", i.e. folding and squashing, will give a great mix quickly. Similarly, this is the way how bakers work on their dough and this coincidence is not by accident.
The point of horseshoe is to demonstrate the phenomenon of "chaos". It is a simple model which reproduces the most complex (symbolic) dynamics found in nature. There is a fundamental theorem in dynamical systems which states that horsheoes are generically (i.e. ALWAYS) present in a system with transversal intersection of heteroclinic or homoclinic manifolds of hyperbolic points. I would suggest looking at Wiggin's Nonlinear dynamical systems and chaos book for an excellent treatment. He develops the horseshoe->then gives the conditions under which a horseshoe is present in a system (called Conley-Moser conditions)->then proves that given a hyperbolic point with transversal intersections of invariant manifolds, the Conley-Moser conditions are satisfied.