What are we solving with row reduction?
The main point of row operations is that they do not change the solution set of the underlying linear system. So when you take a system of linear equations, write down its (augmented) coefficient matrix, and row reduce that matrix, you get a new system of equations that has the same solutions as the original system.
When the coefficient matrix is in reduced row echelon form, it is particularly easy to understand the solution set of the linear system. In particular, you can readily tell whether the system is consistent or inconsistent, and when the system is consistent, you can readily see how a choice for the values of the free (non-pivotal) variables leads to a solution.
I recommend David Lay's book as a textbook for this material.
One thing that sometimes gets lost in the process of Gaussian elimination (the usual algorithm for putting a matrix into reduced row echelon form) is just how similar this process is to the approach, often taught in high school algebra, of solving for one variable in a particular equation and then eliminating its occurrences from all the other equations. When you produce a pivot in a row of the coefficient matrix, this corresponds to solving the equation for that variable. When you add or subtract (multiples of) this pivotal row from the other rows to produce zeros in the pivotal column, the new rows represent the equations you get by substituting in for the pivotal variable. In a sense, Gaussian elimination is just an excellent way to keep track of the process of solving for variables and then eliminating them from the other equations.
Here's an example. We start with the equations
$3x + 6y + 3z = 3$
$2x + y + 7z = 6$.
The corresponding matrix is
$\left[\begin{array}{rrr|r} 3 & 6 & 3 & 3\\\ 2 & 1 & 7 & 6\end{array}\right]$.
Dividing the first row/equation by 3 gives you the equation $x + 2y + z = 1$, which you can think of as $x = 1 - 2y - z$. You can plug that expression into the second equation to get $2(1-2y -z) + y + 7z = 6$, which simpifies to $-3y + 5z = 4$. Notice that when you row reduce by dividing the first row by 3 and then subtracting twice row 1 from row 2, you get the same thing:
$\left[\begin{array}{rrr|r} 3 & 6 & 3 & 3\\\ 2 & 1 & 7 & 6\end{array}\right] \to \left[\begin{array}{rrr|r} 1 & 2 & 1 & 1\\\ 2 & 1 & 7 & 6 \end{array}\right] \to \left[\begin{array}{rrr|r} 1 & 2 & 1 & 1\\\ 0 & -3 & 5 & 4\end{array}\right].$
The point of row reduction is to transform the system into an equivalent form that makes it simpler to deduce key information. In particular the reduced row echelon form immediately reveals fundamental information such as the column space, spanned by the pivot columns, the rank, the nullspace, etc. Generally such information can not be so simply deduced for a matrix in general form. Here is a worked example from Strang's Linear Algebra, 4ed p. 84:
So you have a matrix $A$ and vectors $x,b$, to solve $Ax=b$ you apply row reductions.
So considering a concrete example, if we wanted to solve
$$\left(\begin{array}{cc} 1 & 1 \\ 1 & 2\end{array}\right) x = \left(\begin{array}{c} 5 \\ 8\end{array}\right)$$
We can use the row that changes the bottom row into bottom row minus top:
$$\left(\begin{array}{cc} 1 & 1 \\ 0 & 1\end{array}\right) x = \left(\begin{array}{c} 5 \\ 3\end{array}\right)$$
Which tells us that $x = \left(\begin{array}{c} 2 \\ 3\end{array}\right)$.
The secret is that the second line is actually
$$\left(\begin{array}{cc} 1 & 0 \\ -1 & 1\end{array}\right) \left(\begin{array}{cc} 1 & 1 \\ 1 & 2\end{array}\right) x = \left(\begin{array}{cc} 1 & 0 \\ -1 & 1\end{array}\right) \left(\begin{array}{c} 5 \\ 8\end{array}\right)$$
So a row operation corresponds to multiplying both sides by a matrix - Why is this useful for solving a system? Well it's just the same as when you solve any equation, for example
$$2x+3=7x-4$$
We can add 4 to both sides and then subtract 2x from both to get
$$7=5x$$
That is the meaning of row operations, and why they help to solve a system - it's also interesting to note that if we start out with $AX=I$ and use row operations until it's in the form $IX=B$ then the inverse of $A$ has been found. Viewing the row operations as matrices, lets say it took 3 operations $R_1, R_2, R_3$ then we have $R_3 R_2 R_1 A X = I X = R_3 R_2 R_1 I$ so $A^{-1} = X = R_3 R_2 R_1$.