How far can one get in analysis without leaving $\mathbb{Q}$?
What kind of algebraist "refuses to acknowledge the existence of any characteristic 0 field other than $\mathbb{Q}$"?? But there is a good question in here nevertheless: the basic definitions of limit, continuity, and differentiability all make sense for functions $f: \mathbb{Q} \rightarrow \mathbb{Q}$. The real numbers are in many ways a much more complicated structure than $\mathbb{Q}$ (and in many other ways are much simpler, but never mind that here!), so it is natural to ask whether they are really necessary for calculus.
Strangely, this question has gotten serious attention only relatively recently. For instance:
$\bullet$ Tom Korner's real analysis text takes this question seriously and gives several examples of pathological behavior over $\mathbb{Q}$.
$\bullet$ Michael Schramm's real analysis text is unusually thorough and lucid in making logical connections between the main theorems of calculus (though there is one mistaken implication there). I found it to be a very appealing text because of this.
$\bullet$ My honors calculus notes often explain what goes wrong if you use an ordered field other than $\mathbb{R}$.
$\bullet$ $\mathbb{R}$ is the unique ordered field in which real induction is possible.
$\bullet$ The most comprehensive answers to your question can be found in two recent Monthly articles, by Jim Propp and by Holger Teismann.
But as the title of Teismann's article suggests, even the latter two articles do not complete the story.
$\bullet$ Here is a short note whose genesis was on this site which explains a further pathology of $\mathbb{Q}$: there are absolutely convergent series which are not convergent.
$\bullet$ Only a few weeks ago Jim Propp wrote to tell me that Tarski's Fixed Point Theorem characterizes completeness in ordered fields and admits a nice proof using Real Induction. (I put it in my honors calculus notes.) So the fun continues...
This situation strikes me as about as worthwhile a use of your time as trying to reason with a student in a foreign language class who refuses to accept a grammatical construction or vocabulary word that is used every day by native speakers. Without the real numbers as a background against which constructions are made, most fundamental constructions in analysis break down, e.g., no coherent theory of power series as functions. And even algebraic functions have non-algebraic antiderivatives: if you are a fan of the function $1/x$ and you want to integrate it then you'd better be ready to accept logarithms. The theorem that a continuous function on a closed and bounded interval is uniformly continuous breaks down if you only work over the rational numbers: try $f(x) = 1/(x^2-2)$ on the rational closed interval $[1,2]$.
Putting aside the issue of analysis, such a math student who continues with this attitude isn't going to get far even in algebra, considering the importance of algebraic numbers that are not rational numbers, even for solving problems that are posed only in the setting of the rational numbers. This student must have a very limited understanding of algebra. After all, what would this person say about constructions like ${\mathbf Q}[x]/(x^2-2)$?
Back to analysis, if this person is a die hard algebraist then provide a definition of the real numbers that feels largely algebraic: the real numbers are the quotient ring $A/M$ where $A$ is the ring of Cauchy sequences in ${\mathbf Q}$ and $M$ is the ideal of sequences in ${\mathbf Q}$ that tend to $0$. This is a maximal ideal, so $A/M$ is a field, and by any of several ways one can show this is more than just the rational numbers in disguise (e.g., it contains a solution of $t^2 = 2$, or it's not countable). If this student refuses to believe $A/M$ is a new field of characteristic $0$ (though there are much easier ways to construct fields of characteristic $0$ besides the rationals), direct him to books that explain what fields are.
I did my undergrad project on "Analysis in a rational world", because it seemed like fun. It turns out that using definitions over $\mathbb{Q}$ means you can solve $d^n F/d x^n=G(x,F,dF/dx,\ldots,d^{n-1}F/dx^{n-1})$ with each derivative bijective on almost whatever interval you like. Fun, but not very like real analysis.