How to determine standard equation of a conic from the general second degree equation?

The "complete the square" method allows you to center the conic, i.e. make the linear terms vanish (by a translation, $ax^2+bx+c\to a'u^2+c'$).

To deal with "obliqueness", you need to let the cross term $xy$ vanish, by means of a rotation.

Let $x=cu-sv,y=su+cv$, which expresses a rotation around the origin ($c,s$ denote the cosine and sine of the angle), and let the centered equation be in the form

$$Ax^2+2Bxy+Cy^2=1.$$

Then substituting,

$$(Ac^2+2Bcs+Cs^2)u^2+(-2Acs+B(c^2-s^2)+2Ccs)uv+(As^2-2Bcs+Cc^2)v^2=1.$$

By a suitable choice of the angle, you can achieve

$$-2Acs+B(c^2-s^2)+2Ccs=0$$

and the equation reduces to

$$A'u^2+C'v^2=1.$$

Depending on the signs of $A',C'$, you get an ellipse or an hyperbola. By a further rescaling of the variables, you can obtain the "canonical" forms (circle and equilateral hyperbola)

$$p^2\pm q^2=1.$$

---

To find the suitable angle, use the "double angle" formulas and rewrite

$$(C-A)\sin(2\theta)+B\cos(2\theta)=0,$$ which is easy to solve.

We have $\;ax^2+by^2+2hxy+2gx+2fy+c=0\;$ . Let us write down this quadratic's expanded matrix:

$$A:=\begin{pmatrix}c&g&f\\g&a&h\\f&h&b\end{pmatrix}$$

Observe that firs row is [free coefficient , half coefficient of x, half coefficient of y], the second row is [hlaf coef. of x, coeff. of x^2 , half coef of xy] and etc.

The corresponding reduced matrix is

$$A':=\begin{pmatrix}a&h\\h&b\end{pmatrix}$$

We have, of course, that both matrices are symmetric. Let us now define for a square matrix $\;B\;$ :

$$\color{red}{r(B)}:=\text{ the rank of}\;\;B\;,\;\;\color{red}{|B|}:=\det B\;,\;\;\color{red}{\sigma(B)}:=\text{ the signature of}\;\;B$$

where signature = the difference between the number of positive elements with the number of negative elements in the main diagonal of the matrix.

Then we have, for example (the whole table is way too long):

$$\begin{cases}r(A)=3\;,\;\;r(A')=2\;,\;\;|A|,A'|>0\;\color{red}\implies\;\text{Empty set}\;(\emptyset)\\{}\\r(A)=3\;,\;\;r(A')=2\;,\;\;|A|<0,\;|A'|>0\;\color{red}\implies\;\text{Ellipse}\\{}\\r(A)=3\;,\;\;r(A')=2\;,\;\;|A'|<0\;\color{red}\implies\;\text{Hyperbola}\\{}\\r(A)=3\,,\,\,r(A')=1\;\color{red}\implies\;\text{Parabola}\end{cases}$$

and etc. The signature kicks in for one pair of parallel lines ($\;\sigma(A)=0\;$ and the ranks one less than the maximal possible), and for empty set ($\;\sigma(A)=2\;$ and the ranks as before)

The above is related to bilinear forms and their close relatives, quadratic forms, Sylvester's Inertia Theorem and etc. It is a little advanced material in linear algebra and it is usually covered, as far as I know, in a second course in this subject at undergraduate level.