What is the simplest way to rigorously define division (why does the standard algorithm work)?

You might understand the division algorithm better by using simple equality:

$$\frac{x^3+6x^2+3x-8}{x^2+x-2}$$ $$= \frac{(x^3+6x^2+3x-8)-(x^2+x-2)x}{x^2+x-2}+\frac{(x^2+x-2)x}{x^2+x-2}$$ $$= \frac{5x^2+5x-8}{x^2+x-2}+x$$

We specifically chose $x$ to add and subtract from the expression, so as to cancel the $x^3$ on the left. The purpose of the algorithm is to reduce the degree of $p(x)$, eventually to a lower degree than $q(x)$.

Let $p(x) = p_mx^m+p_{m-1}x^{m-1}+\cdots+p_1x+p_0$,

and $q(x) = q_nx^n+q_{n-1}x^{n-1}+\cdots+q_1x+q_0$,

with $m \geq n$ and $p_m\neq0\neq q_n$. Then

$$\frac{p(x)}{q(x)} = \frac{p_mx^m+p_{m-1}x^{m-1}+\cdots+p_0}{q_nx^n+q_{n-1}x^{n-1}+\cdots+q_0}$$ $$= \frac{(p_mx^m+p_{m-1}x^{m-1}+\cdots+p_0)-\frac{p_mx^m}{q_nx^n}(q_nx^n+q_{n-1}x^{n-1}+\cdots+q_0)}{q_nx^n+\cdots+q_0}+\frac{p_mx^m}{q_nx^n}$$ $$= \frac{(p_mx^m+p_{m-1}x^{m-1}+\cdots+p_0)-p_mx^m-\frac{p_m}{q_n}q_{n-1}x^{m-1}-\cdots-\frac{p_m}{q_n}q_0x^{m-n}}{q_nx^n+\cdots+q_0}+\frac{p_m}{q_n}x^{m-n}$$ $$= \frac{p_{m-1}x^{m-1}+\cdots+p_0-\frac{p_m}{q_n}q_{n-1}x^{m-1}-\cdots-\frac{p_m}{q_n}q_0x^{m-n}}{q_nx^n+\cdots+q_0}+\frac{p_m}{q_n}x^{m-n}$$

The degree of the numerator has been reduced from $m$ to $m-1$. It should be clear that $\frac{p_m}{q_n}x^{m-n}$ is the only term, exactly the term that can be subtracted to cancel $p_mx^m$.

Division is basically partitioning an expression/value into (smaller) components. It is the number of times that the components can be contained within the expression/value.

Let's convert base $10$ to a variable $x$. So $903$ becomes $9x^2+3$ and $12$ becomes $x+2$. Then we get $$\begin{align}9x-18\\\text{____________}\\x+2\text{/}\,\,9x^2+0x+3\\\hspace{-3cm}9x^2+18x\\\text{____________}\\-18x+3\\-18x-36\\\text{________}\\39\end{align}$$

Note: Feel free to edit this attempt at writing long division with $\LaTeX$.

Now we would usually stop here, giving $$903=9\cdot10-18+\frac{39}{12}=75.25$$ But if we continue, we would arrive at $$9x-18+39x^{-1}-78x^{-2}+156x^{-3}-312x^{-4}+\cdots$$ or $$9x-18+\frac{39}x\sum_{k=0}^\infty\left(-\frac2x\right)^k$$ and since $|-2/x|=0.2<1$, we can invoke geometric series. Thus the expression becomes $$9x-18+\frac{39}{x\left(1+\frac2x\right)}=9\cdot10-18+\frac{39}{10\cdot1.2}=75.25,$$ as before.

In essence, this is just a rather interesting way to write $\frac14$ as a geometric series!

TheSimpliFire has already dealt with the infinite series part of your question. Therefore, I shall restrict myself to the 'rigorous' definition of division.

One, potential approach (as you discovered) is to define division as an infinite series. Note however, that the steps to do so may implicitly involve division.

As it turns out, in rigorous math (say, abstract algebra), the concept of division (and subtraction) is usually never explicitly defined. Usually , we just define multiplication and then define multiplicative inverses (think, reciprocals) as follows: I For each, $a \in \mathbb{R^*}$, there exists $(1/a) \in \mathbb{R^*}$ such that $a \cdot (1/a) = 1$. (Here, we have defined the multiplicative inverse on the set $\mathbb{R^*}$ of non-zero real numbers.)

Of course $1$ is already defined by $1 \cdot a = a$ with the proper conditions appended.

Then you can 'define' division of $a$ by $b$ as multiplication of $a$ by $(1/b)$.

Multiplication itself, is defined recursively using addition; while addition can be defined by extending it from the natural numbers while defining them using Peano's axioms.

Note that all definitions of the elementary operations are somewhat revised everytime we proceed to a bigger set (say, from $\mathbb{Q}$ to $\mathbb{R}$).

Also, all operations are not defined for all sets. For example, division cannot be defined on the set $\mathbb{N}$, of natural numbers.

You can read up on set theory, abstract algebra and real analysis if you want to know more.