What Gauss *could* have meant?
The Wikipedia entry, and a few other places that mention this, give as reference the popular book of John Derbyshire on the Riemann hypothesis. The book itself mentions this in the spirit of an apocryphal tale, with the passing remark that "I wouldn't put it past him", and gives no reference at all.
It's probably best to treat this story with utmost suspicion until better evidence turns up. This is the kind of tale that proliferates in mathematics because on the one hand it adds to the legend of the people we admire, whom we suspend critical judgement for in favour of awe and wonder, and on the other hand it plays on the insecurities of the average aspiring mathematician.
For people with such pretense of belief in reason, mathematicians are a little too often purveyors of rumours and hearsay themselves.
If you look at the formula geometrically (and draw the unit circle), you'll see immediately that $e^{\pi i}=-1$, since it's half a circle from the origin.
I don't see any other way for this to be all too apparent.
I have to agree with Prometheus, as this is probably bunk. However, I think there is an interesting interpretation to be gleaned from this Gauss-attributed comment.
I've had the distinct pleasure of talking with UChicago's Professor Benson Farb, an esteemed geometer and one of my personal idols, on two occasions.
The first time I talked with him was through a Skype call. The conversation was centered around what I could do, as a budding mathematician, to start heading in the right direction. I don't recall his exact words, but one of his main points was that I should focus on having a strong foundation before getting into advanced topics. I truly agree with this idea; during my sophomore year, I'd work for at least four hours every day just on calculus. Now, I can not only do geometry, but I can do it well because of my stronger background.
The second time I met him was at a conference in Columbus, Ohio. His lecture (link goes to YouTube) was geared toward undergraduate mathematicians. At 45:30, he gives a variation on the same theme: working with "the basics" helped him become successful.
I also had the opportunity to Skype with another UChicago faculty member, Professor Calegari, while I was at an international conference in Canada. Unsurprisingly, he had the same message: foundations are good.
I think we become, as Farb puts it, "enamored with machinery." We tend to focus on the cool things without paying due attention to the smaller gears to the larger machine. Perhaps Gauss, or his impersonator, meant that if we learned that $e^{i\pi}=-1$ before we learned the series expansions for $x\mapsto e^x$, $x\mapsto\sin(x)$, and $x\mapsto\cos(x)$, then we wouldn't be able to function as well, as mathematicians?