What exactly is nonstandard about Nonstandard Analysis?
The term "nonstandard" refers to nonstandard analysis using a nonstandard model. In nonstandard analysis, one usually intends to study the real numbers (and their functions and relations). The real numbers under certain operations model a field. In nonstandard analysis, the intended model consists of the real numbers, while the model used for study of the real numbers usually consists of a hyperreal field. Hyperreal fields are not isomorphic to the real number field. We can see this just by looking at finite hyperreal structures, that is just finite numbers, infinitesimals, and numbers infinitely close to finite numbers.
Given a hyperreal field with only finite elements, we do have a homomorphism from those finite hyperreals to the reals, at least once we take care of division. So, in the finite case, we can regard the "standard part" or "shadow" of a hyperreal number as a function, but we cannot correctly obtain an inverse function from the reals to the finite hyperreals. We also have a homomorphism from basically any structure of finite hyperreals to a given real structure.
For example, if we consider $(\mathbf{H}, +', *', -', /')$, that is the finite hyperreals under hyperreal addition, multiplication, subtraction, and division (where the denominator is not an infinitesimal) we have a map $\DeclareMathOperator{\SH}{SH}\SH$ to $(\mathbf{R}, +, *, -, /)$ such that
$$\begin{array}{} \SH(h&+'&i)&=&(\SH(h)&+&\SH(i))\\ \SH(h&*'&i)&=&(\SH(h)&*&\SH(i))\\ \SH(h&-'&i)&=&(\SH(h)&-&\SH(i))\\ \SH(h&/'&i)&=&(\SH(h)&/&\SH(i))\\ \end{array}$$
and for all finite hyperreal numbers $h$, there exists a real number $r$ such that $\SH(h)=r$.
No inverse of the shadow function from the reals to the finite hyperreals exists, or in other words no function $I$ exists such that $I(\SH(h))=h$ holds true. This comes perhaps as easiest to see if you consider that the shadow of each infinitesimal number equals $0$. But, $0$ does not associate with a unique hyperreal infinitesimal, since we could associate $0$ with one of an infinity of positive infinitesimals, or one of an infinity of negative infinitesimals in rather the same way.
See Pete L. Clark's last two comments below also.
Also, the Dedekind completeness property fails for the hyperreals, which indicates the hyperreals as a non-isomorphic structure to the reals under various operations. Keisler's text referenced by Brian M. Scott makes for a good reference to read, and it has some good examples.
It might help to understand a tiny little bit of the background leading Abraham Robinson to developing his model of nonstandard analysis. There have been several approaches to incorporating infinitesimals into analysis. One such approach was developed by Schmieden and Laugwitz. They considered sequences of real numbers under the equivalence relation where two sequences are equal if from some point onwards all their coordinates agree. This leads to an extension of the real numbers but does not yield a field (but rather a ring with zero divisors). Nonetheless, one can still do quite a lot of analysis in this larger ring which includes proper infinitesimals.
Roughly the same time Skolem had used fundamental results in logic (model theory) to construct models of the natural numbers that included infinitely large natural numbers. These models were not intended to have any applications but rather used for logic oriented investigations. Skolem called these models of the naturals nonstandard models of arithmetic.
Robinson was well aware of the work of Schmieden and Laugwitz and of Skolem's work. Robinson's great contribution was in the realization that the Schmieden-Laugwitz model can be greatly improved utilizing Skolem's techniques. It is thus almost certain that Robinson chose the term nonstandard analysis due to the existing nonstandard models of arithmetic.
"Nonstandard" is really referring to the nonstandard numbers, including the infinitesimal numbers and hyperreal numbers. Foundationally, it can be shown that if the structure we call "the real numbers" and denote $\mathbb{R}$, exists, then so does another structure, $\mathbb{R}^{*}$ that, in certain useful ways, behaves just like that structure, but has an element $\epsilon$ such that $0<\epsilon < 1/n$ for every positive integer $n$. This new element is called nonstandard, or more specifically infinitesimal, since it is nonzero yet smaller than every positive standard real number.
From the existence of just this one nonstandard element can be proven the existence infinitely many more, even hyperreals. For example, since the sentence $\forall x\in \mathbb{R}, \exists y \in \mathbb{R}[0<y<x \vee x<y<0]$ holds of $\mathbb{R}$, it also holds of $\mathbb{R}^{*}$, so we get infinitely many infinitesimals (after applying that sentence to $x=\epsilon$). Similarly, the sentence $\forall x \forall y [x<y \rightarrow 1/x > 1/y]$, with $x=\epsilon$ and $y=1/n$ shows that $1/\epsilon$ (or the reciprocal of any infinitesimal) is bigger than any standard real; i.e. it is hyperreal. These two sentences are examples of the "useful ways" that $\mathbb{R}^{*}$ behaves like $\mathbb{R}$. (This is really the transfer principle.)