How do you explain the use of studying higher mathematics to non-mathematicians?
One thing I've noticed about mathematics (and, to an extent, the sciences*) is that they train you to trust a large number of deductive links over a small number of intuitive ones. In his Gödel, Escher, Bach, Doug Hofstadter gives a proof of the commutativity of addition, starting from some basic postulates (not Peano, but something akin to Peano, if I recall correctly). It occupies something like fifty-six separate lines, almost every one relying in some way on the previous ones.
That is not something that would convince most people. I assume most people, to the extent that they've thought about addition, have never thought about it not commuting, and if they had, they would treat the fact of it actually commuting as something so obvious as to defy proof. But in addition, they would treat fifty-six steps as being way too long to be convincing.
Mathematics trains us to consider fifty-six steps as no less rigorous than six, even if we can hold all of the latter in our head, but not the former. Part of the reason is that the fifty-six steps have an overarching structure, with maybe six "paragraphs" that each have their own "intent." It is at that level that the intuition of commutative addition is represented, even though the rigor is represented at a lower, line-by-line level.
Another aspect of mathematics that does not occur to many people is the degree to which it unifies concepts. This too is something it shares to an extent with the sciences. Whether it is something as trivial as unifying combinatorics and probability, or as deep as the bond between Fermat's last theorem and modular forms, there is an interconnectedness between things that is fundamental to doing mathematics. More often than in any field, I feel, it achieves that interconnectedness. It is the substance of analogies—what are the connections, say, between the colonization of China and that of the U.S. colonies?—made exact, made (again) rigorous.
Finally, I remember reading one of Isaac Asimov's essays in the Magazine of Fantasy and Science Fiction, in which he describes his conception of science and mathematics: like a fractal, so that no matter how closely, how finely you look at it, you see as much complexity at the fine level as you saw at the coarse level. That is something you can appreciate, I think, only after doing lots and lots of mathematics. The fact that after thousands of years of humans doing mathematics, we haven't come any closer to exhausting the field—that we have in fact much more to do than at any other time in history—is something that, when I think hard on it, seems almost miraculous. Why should something so pure, so fundamental, be at the same time so rich? That's something I don't have anything approaching a good answer for, but I appreciate it.
*It's been pointed out, in the comments, that this is not something shared only by mathematics and the hard sciences. My attribution was based only on my limited experience with stuff outside that arena. I should not have implied the exclusion of other fields.
I will try to answer the hidden presumption that this question takes for granted rather than the popular answer or list of quotes from great mathematicians.
Mathematics is a tool. One of many in your arsenal. It is based on deductive logic but it is not a synonym of deductive logic, rather a broad subset.
As for the question of elegance or beauty, remember that form follows the function. Beauty and elegance is derived by the purpose. Nothing is elegant in a vacuum nor does a tool automatically gain elegance by majority vote.
Other than that, Maths is great. Even greater is that you enjoy it. Just keep the automagical presumptions of elegance at bay and prove it's elegance by application if you can.