Differential equations: how does separation of variables really work?
This text has been copied from http://kevinboone.net/separation_variables.html
I have edited the equations only so that they can be read easily, but I do not want to take any credit for the original text.
Copyright remains as stated on the website: Copyright © 1994-2013 Kevin Boone. Updated Oct 02 2013 © 1994-2013 I have not asked the copyright holder's permission.
Differential equations: how does separation of variables really work?
One of the simplest forms of differential equation to solve is that in which the variables can be separated on either side of the equality. Such equations occur frequently in science and engineering, and are usually the first problems of this type to be taught to students.
Some of the operations that textbooks and classes teach to students, as part of the process of solving separable-variable equations, appear to be mathematically highly dubious. Students are typically taught to treat the differential operator $\frac{dy}{dx}$ as if it were a simple ratio of two quantities, whose individual terms can be multiplied and divided independently. However, $\frac{dy}{dx}$ is not a ratio, and is not clear why these procedures work. Worse, it is unclear whether it is always safe to treat the differential operator in such a cavalier fashion, or whether it is only safe for particularly classes of problem.
This situation irritated me when I was first taught calculus more than thirty years ago, and it continues to irritate me to this day.
In this article I will attempt to describe how we can deal with separation of variables without playing fast and loose with differentials. In doing so, I hope to provide some insight into what is going on behind the scenes when we follow the method that is usually taught. The arguments in this article apply also to other procedures in calculus that appear to rely on treating a differential operator as a ratio, such as integration by U-substitution.
I'll start by describing the usual textbook procedure for solving a simple differential equation with separable variables, and the explain what's wrong with it, and how it can be put right.
I want to point out right from the start that this article is really about the philosophy of mathematics. You can follow the textbook method and get the right answer without understanding any of this stuff.
The textbook approach
Consider the following simple differential equation for x and y $$\frac{dy}{dx} = \frac{x^2}{1-y^2}$$ Our job is to find a relationship between $x$ and $y$ that does not contain a derivative term, that is, to solve for $y$ in terms of $x$.
The usual textbook approach to solving an equation of this type is to separate the variables — putting all the $x$ terms on one side, and the $y$ terms on the other. The $x$ terms have to include $dx$, and the $y$ terms $dy$. We can multiply both sides by $dx$ and by $1-y^2$, and this gives us: $$(1 - y^2)dy = x^2 dx$$ But what does it mean to 'multiply by $dx$?' $dx$ is not a quantity — it is a component of the operator $\frac {d}{dx}[f(x)]$. While it is true that the '$d$' terms do, in some senses, represent small values ('infinitesimals', in the Leibnitz formulation), the operator itself is a limit expression (maybe; see the closing remarks for discussion) — it denotes the limiting value of one infinitesimal as the other approaches zero. The components of $\frac{dy}{dx}$ have little significance on their own, and they certainly don't form an arithmetical ratio.
Leaving that problem aside for now, the textbook procedure now typically calls for us to write an integration sign in front of each side (whatever that signifies), to create a pair of integrals:
$$\int (1 - y^2 dy = \int x^2 dx$$ We now have something that is mathematically well-formed, but we've got to it by a peculiar process. Like the differential operator, the indefinite integral operator $\int dx$ is something that is meaningful in a particular form; the ∫ doesn't have much meaning on its own.
Be that as it may, integrating both sides with respect to their independent variables gives:
$$y - \frac{y^3}{3} = \frac {x^2}{3} + C$$
With a bit of tidying up:
$$3 y - y^3 = x^2 + C$$ This is a cubic equation in y, and not easy to juggle into a straightforward relationship of the form $y=f(x)$. Still, we have a solution (or, rather, a family of solutions) to the original differential equation; the calculus is done, and the rest is algebra.
So what's the problem?
Most textbooks and classes gloss over the fact that $\frac{dy}{dx}$ is not a ratio of two quantities, and treat the individual terms as though they can be multiplied, divided and canceled at will. Some authors do at least point out the problem, but usually to say "$\frac{dy}{dx}$ is not a ratio, but it can be helpful on occasions to treat it as if it were." It's never very clear what these occasions are. Sometimes you'll come across statements such as "This procedure is justified by the chain rule" or "This procedure is justified by the Fundamental Theorem of Calculus." Maybe that's true, but how is it justified? You'll sometimes see expressions in $dx$ or $dy$ explained away with hand-waving expressions like "formal form" or "infinitesimal form". But what do those terms really mean? By leaving these questions unanswered, students are left with no way to figure out whether the approach being explained is generally applicable, or works only for a limited class of problems.
Another attempt
Let's see if we can't improve the formulation of the separation of variables procedure, and avoid some of these ugly mathematical kludges. Starting with the original differential equation:
$$\frac{dy}{dx} = \frac{x^2}{1-y^2}$$
We can rearrange to give:
$$(1-y^2)\frac{dy}{dx} = x^2$$
Doing this keeps $\frac{dy}{dx}$ intact, for now.
Now let's take the indefinite integral of each side with respect to $x$. This is a legitimate thing to do, as we're using the same independent variable in the two integrals, even though the LHS is a function of $y$ as well as $x$:
$$\int(1-y^2) \frac{dy}{dx} dx=\int x^2 dx$$
From here, we could "cancel out" the $dx$ terms to leave:
$$\int(1-y^2){dy}=\int x^2 dx$$
and this is the same separated form we arrived at earlier.
But what does it mean to "cancel out" the $dx$ terms? Is doing this any better than manipulating $dy$ and $dx$ independently? Not really — all we've done is postpone the application of a kludge — we're still going to have to treat $\frac{dy}{dx}$ as a ratio — we're just doing it a bit later. The $dx$ in the integration operator is not the same quantity as the $dx$ in the differentiation operator, if it is even a quantity at all.
Giving the lack of mathematical rigour, it can be somewhat surprising that these procedures do actually work — in this particular kind of problem, at least, manipulating $dy$ and $dx$ independently does allow the correct solution to be reached.
But how?
"Cancelling $dx$" as a shortcut to applying the chain rule
To figure out what's going on here, we need to think about what an indefinite integral actually means. In essence, when we look to evaluate
$$\int(1-y^2) \frac{dy}{dx} dx$$
what we're really asking is "What can be differentiated with respect to $x$, to give $(1-y^2) \frac{dy}{dx}$?
It's an odd question, on the face of it, because aren't particularly used to see the results of a differentiation having differential terms in them. There's certainly no problem finding something that will differentiate to $1-y^2$ : straightforward integration gives us $y-\frac{y^3}3 + C$ , for any constant $C$. Knowing this, it is possible to find an expression intuitively that will differentiate to $(1-y^2) \frac{dy}{dx}$ . Such an expression is $y-\frac{y^3}3$ , where $y$ is a function of $x$, and we differentiate in terms of $x$, not $y$. To perform this differentiation we need the chain rule:
$$\frac{du}{dx}=\frac{du}{dy}\frac{dy}{dx}$$
If we take $u = y-\frac{y^3}3$ , where $u$ is a function of $x$, then its derivative with respect to $x$ is:
$$\frac d{dx} [y-\frac{y^3}3] = \frac d{dy} [ y-\frac{y^3}3] . \frac {dy}{dx} = (1 - y^2) . \frac {dy}{dx}$$
In other words, the LHS of
$$\int(1-y^2){dy}=\int x^2 dx$$
can be replaced to give:
$$(y-\frac{y^3}3){dy}=\int x^2 dx$$
This is exactly where we got to by "cancelling out" the $dx$ terms and performing the integration of the LHS with respect to $y$, as explained above. We could then proceed by straightforwardly integrating the RHS.
The above reasoning shows that the "cancelling out" of the $dx$ terms was justified, in the sense that doing it led to the same answer that could be obtained by intuitively finding an expression that could be differentiated to give the integrand in question. In practice, however, we need something more general — something that justifies the operation used, and can be applied routinely in problems of this short. For that, we need to show with reasonable rigour that
$$\int f(y) \frac{dy}{dx} dx =\int f(y) dy$$
That is, we need to show that "cancelling the $dx$ terms" is mathematically valid.
To do that, we'll start with the chain rule again:
$$\frac{du}{dx}=\frac{du}{dy}\frac{dy}{dx}$$
Then let $u = \int f(y) dy$
Substituting into the chain rule:
$$\frac{d}{dx}\left(\int f(y) dy\right)=\frac{d}{dy}\left(\int f(y) dy\right)\frac{dy}{dx}$$
The first term on the RHS is simply a differentiation of an integration with respect to the same variable, $y$, and so the equation reduces to:
$$\frac{d}{dx}\left(\int f(y) dy\right)=f(y) . \frac{dy}{dx}$$
Now if we integrate both sides with respect to $x$, we get
$$\int \left[ \frac{d}{dx}\left(\int f(y) dy\right)\right]dy=\int \left[ f(y) \frac{dy}{dx} \right ] dx$$
As the RHS is just the integral of a derivative with respect to the same variable, the equation reduces to:
$$\int f(y) dy=\int f(y) \frac{dy}{dx} dx$$
which is what we set out to prove. So where does that all get us? In summary, when we split up $\frac{dy}{dx}$ into separate terms and manipulate those terms separately, what we're really doing is integrating both sides of the equation with respect to the same variable, and then applying the chain rule to remove the differential term from the integrand. Treating $dx$ and $dy$ as elements of a ratio is sloppy, but it is mathematically sound to do so in cases where the chain rule would be capable of yielding the integrands in question. And, of course, it's a lot more convenient than applying the chain rule explicitly every time.
Closing remarks
The meaning of $\frac{dy}{dx}$ is itself problematic. Traditionally it has been seen as an operator, derived by considering infinitesimal changes in one variable with respect to another. However, this formulation is not without its problems — most prominently that there is no 'infinitesimally small' real number. The set of real numbers is, by definition, infinitely sub-dividable into smaller sets. To some extent, problems understanding what 'infinitesimal' meant in the context of real numbers led to derivatives being treated as limiting expressions, rather that ratios of infinitesimals. However, there are other ways to understand a derivative, including Abraham Robinson's rigorous definition of infinitesimals in terms of hyperreal fields. Some of these formulations might allow $dy$ and $dx$ to be treated independently in some circumstances.
Nevertheless, it seems to me that explaining the methodology of separation of variables and integration by substitution as a short-cut to the application of the chain rule is more likely to be understood by students, than a highly technical discussion of the meaning of a differential.
As I see it, the procedure can be described by the following steps $$ \frac{dy}{dx}=\frac{g(x)}{h(y)}\tag1 $$ multiplying $h(y)$ and $dx$ on both sides as if both were regular real valued expressions $$ h(y)dy=g(x)dx\tag2 $$ applying integration on both sides to have $$ \int h(y)dy=\int g(x)dx\tag3 $$ and using antiderivatives to get to $$ H(y)=G(x)+c\tag4 $$ and hopefully $H$ is injective so we can solve for $y$ $$ y=H^{-1}\left(G(x)+c\right)\tag5 $$ which may be subject to different constraints on its domain depending on the details in the steps above.
The problem is of course that $dx$ and $dy$ have not been defined as independent real valued expressions. Thus step $(2)$ above may contain an undefined non-sensical expression.
By definition $\frac{dy}{dx}=y'(x)$, but other than that we know little about $dx$ and $dy$. In a historical context, Leibniz and his followers used them and assigned meaning to them individually, but a struggle over their actual meaning and the foundations of calculus went on for centuries after that.
One way to resolve this is to replace $dx$ and $dy$ by $x'$ and $y'$ defined as $x'=1$ and $y'=y'(x)$. Then we have $$ \frac{dy}{dx}=\frac{y'}{x'} $$ where the right hand side is now a fraction between actual real valued expressions. Thus the first steps can be written as $$ \frac{y'}{x'}=\frac{g(x)}{h(y)}\tag1 $$ and multiplying by $h(y)$ and $x'$ on both sides we get $$ h(y)y'=g(x)x'\tag2 $$ which now is meaningful and leads to the intermediate step $$ \int h(y)y' dx=\int g(x)x'dx\tag{2.5} $$ and by applying the substitution $u=y$ on the LHS and noting that $g(x)x'=g(x)$ since $x'=1$ on the RHS this leads to the same step $(3)$ as the original version, namely $$ \int h(y)dy=\int g(x)dx\tag3 $$ So by comparison, the original method is a shortcut to the exact same step $(3)$ as this more rigorous and detailed version.
The whole thing depends on the substituion method being correct, namely $$ \int h(y)y'dx=\int h(y)dy $$ The above is of course correct, since the RHS evaluates to $H(y)+c$ which has derivative $h(y)y'$ (by the chain rule) so both sides are in fact antiderivatives of the same integrand, which proves that the substitution method is valid.