Why Study Quadratic Forms?
Quadratic forms are indeed important in number theory (since you mention Gauss, this should appear very plausible then), but there are many other areas. I just mention a few links from this site:
What's the use of quadratic forms?
Intuition/meaning behind quadratic forms
Applications of quadratic forms
Applications of simultaneous diagonalization of quadratic forms
How does the theory of the quadratic number fields relate to the quadratic forms?
There is so much to say that it is difficult to decide where to start. For sake of simplicity, I will restrict myself to characteristic different from $2$.
Perhaps one of the most useful property of quadratic forms is that quadratic forms (up to hyperbolic ones) form a ring, which is pretty well understood now, and in which it is easy to compute.
One main feature of quadratic forms is the following: in many cases, you can attach a quadratic form to a mathematical object, which only depends on the isomorphism class of your object.
Let me give you a concrete example. If $E$ is a finite dimensional unital associative $F$-algebra, not necessarily commutative ($F$ is a field), for $x\in E$, you can set $tr_{E/F}(x)=tr(\ell_x)$, where $\ell_x$ is the trace of left mulitplication by $x$ in $E$. Then you have a quadratic form $q_E:x\in E\mapsto tr_{E/F}(x^2)\in F$
We then have the property $E\simeq E'\Rightarrow q_E\simeq q_{E'}$.
The isomorphism class of $q_E$, called the trace form of $E$, is then an invariant of the isomorphism class of $E$.In particular, if two algebras have non-isomorphic trace forms, they are not isomorphic.
If you have an involution $\sigma$ on $E$, you can take the quadratic form $q_{E,\sigma} :x\in E\mapsto tr_{E/F}(x\sigma(x))\in F$, whose isomorphism class is an invariant of the isomorphism class of $(E,\sigma)$ .
The nice thing about quadratic forms is that they come with loads of invariants (rank, signature, Clifford invariant, Arason invariant, and so on...) that you can use to do computation.
The trace form has been extensively studied. Let me give you some nice applications:
If $f\in\mathbb{Q}[X]$ is a monic non constant polynomial, and if $E_f=\mathbb{Q}[X]/(f)$, then the signature of $q_{E_f}$ is the number of real zeros of $f$.
If $E/F$ is a finite Galois extension with Galois group $G$, and if $\tilde{G}$ is a certain central extension by $\{\pm 1\}$ that I don't want to describe here, then there exists a Galois extension $\tilde{E}$ of group $\tilde{G}$ containing $E$ as a subextension if and only if $w_2(q_E)=(2,\det(q_E))\in Br(F)$, where $w_2$ is the second Stiefel Whitney class
The trace form of division algebras which are finite dimensional over their center has been useful to prove some structure theorems in low dimension of the type "if the trace form of the algebra has a certain shape, then the division algebra is cyclic/a tensor product of two quaternion algebras", and so on.
Trace forms of algebras with involutions where used by Berhuy, Monsurro and Tignol to provide the first examples of adjoint groups of type $C_n$ and new examples of adjoint groups of type $D_n$ defined over an arbitrary field $F$ which are not rational (that is, such that the $F$-points cannot be parametrized by rational fractions)
Very recently, trace forms have been proved useful in enumerative geometry such as "count the number of lines meeting $4$ general lines defined over $k$ in $\mathbb{P}^3_k$", where $k$ is not necessarily algebraically closed (Srinivasan, Wickelgren, Wendt...)
Other applications (not involving necessarily trace forms):
Pfister proved using the machinery of quadratic forms that for all n$\geq 0$, the product of the sum of $2^n$ squares is again a sum of $2^n$ squares.
Integral quadratic forms are closely related to integral lattices, and are used in coding theory (especially for Wifi), cryptography, computer sciences.
I could go on and on...