What's a Turing machine?
The reason that Turing Machines are a big deal has to do with the study of classical Computing Science or Theory of Computation type stuff. It's basically about analyzing the general properties of a computer, such as what theoretical abilities and limitations a computer has, as well as what we mean when we talk about "computing" something.
One example of something that one might study using Turing Machines is The Halting Problem. While this problem is something of an academic exercise, it has easily tangible real-world implications. Why not write a debugger that will simply tell you whether or not your program contains any infinite loops? The Halting Problem establishes that solving this problem for the general case is impossible.
The study of Turing Machines also lends itself to studying language grammars and classes of thereof, which leads into programming language development. The term "regular expressions" comes about because they are a regular grammar, and the study of these grammars (part of Theory of Computation) will tell you more about exactly what kinds of problems regular expressions can solve and what they can't. For example, a traditional regular expression syntax won't be able to solve the following problem: parse some number N of 'a' chars in input, and then parse the same number N of char 'b'.
If you're interested in a good text about this sort of thing, check out Introduction to the Theory of Computation by Michael Sipser. It's good.
The Turing machine is a theoretical computing machine invented by Alan Turing to serve as an idealized model for mathematical calculation, basically its a simple form of computer, its composed by a tape (a ribbon of paper), has a head that can read the symbols, write a new symbol in place, and then move left or right.
The Turing machine is said to be in a certain state, and then a program is a list of transitions, having a current state and a symbol under the head, what should be written on the tape, what would be the next state, and where the head should move.
Here is a Basic Turing Machine, implemented in JavaScript...
And a sketch:
My IBM PC is all I need to do my computation!
Something that others haven't pointed out: Your IBM PC is a Turing machine. More precisely, it is equivalent to it, in the sense that anything your PC can do, a Turing machine can do, and anything a Turing machine can do, your PC can.
Specifically, a Turing machine is a model of computation that completely captures the notion of computability, while remaining simple to reason about, without all the specific details of your PC's architecture.
The (generally accepted) "Church-Turing thesis" asserts that every device or model of computation is no more powerful than a Turing machine. So, many theoretical problems (e.g. classes like P and NP, the notion of "polynomial-time algorithm", and so on) are formally stated in terms of a Turing machine, although, of course, they can be adapted to other models as well. (For example, sometimes it can be convenient to think of computation in terms of the lambda calculus, or combinatory logic, or whatever... they are all equivalent in power to each other, and to your IBM PC as well.)
So there you go: people talk about Turing machines because it is a precise and full specified way to say what a "computer" is, without having to describe every detail of the CPU's architecture, its constraints, and so on.