A remark of Connes
...as soon as you have a non-standard number, you get a non-measurable set.
Every nonstandard natural number $N$ gives rise to a nonprincipal ultrafilter $U$ on $\mathbb{N}$, by saying that a set $X\subset\mathbb{N}$ is in $U$ if and only if $N\in X^*$, the nonstandard analogue of $X$. In other words, the ultrafilter says $X$ is large if it expresses a property that the nonstandard number $N$ has. We may regard $U$ as a subset of $2^\mathbb{N}$, which carries a natural probability measure. But a nonprincipal ultrafilter cannot be measurable there, since the full bit-flipping operation, which is measure-preserving, carries $U$ exactly to its complement, so $U$ would have to have measure $\frac12$, but $U$ is invariant by the operation of flipping any finite number of bits, and so must have measure $0$ or $1$ by Kolmogorov's zero-one law. (See also this article by Blackwell and Diaconis proving the same fact.)
...every set you can name is measurable.
Another way of saying that a set is easily described is to say that it lies low in the descriptive set-theoretic hierarchy, and the lowest such sets are necessarily measurable. For example, every set in the Borel hierarchy is measurable, and the Borel context is often described as the domain of explicit mathematics. Under stronger set-theoretic axioms, such as large cardinals or PD, the phenomenon rises to higher levels of complexity, for under these hypotheses it follows even that all sets in the projective hierarchy are Lebesgue measurable. This would include any set that you can define by quantifying over the reals and the integers and using any of the basic mathematical operations.
Connes' critique was recently analyzed by Kanovei, Katz, and Mormann in this article in Foundations of Science (see also arXiv 1211.0244). Here is the abstract:
We examine some of Connes' criticisms of Robinson's infinitesimals starting in 1995. Connes sought to exploit the Solovay model S as ammunition against non-standard analysis, but the model tends to boomerang, undercutting Connes' own earlier work in functional analysis. Connes described the hyperreals as both a 'virtual theory' and a 'chimera', yet acknowledged that his argument relies on the transfer principle. We analyze Connes 'dart-throwing' thought experiment, but reach an opposite conclusion. In S, all definable sets of reals are Lebesgue measurable, suggesting that Connes views a theory as being 'virtual' if it is not definable in a suitable model of ZFC. If so, Connes' claim that a theory of the hyperreals is 'virtual' is refuted by the existence of a definable model of the hyperreal field due to Kanovei and Shelah. Free ultrafilters aren't definable, yet Connes exploited such ultrafilters both in his own earlier work on the classification of factors in the 1970s and 80s, and in Noncommutative Geometry, raising the question whether the latter may not be vulnerable to Connes' criticism of virtuality.
We analyze the philosophical underpinnings of Connes' argument based on Goedel's incompleteness theorem, and detect an apparent circularity in Connes' logic. We document the reliance on non-constructive foundational material, and specifically on the Dixmier trace (featured on the front cover of Connes' magnum opus) and the Hahn-Banach theorem, in Connes' own framework. We also note an inaccuracy in Machover's critique of infinitesimal-based pedagogy.
A brief review of Kanovei-Shelah is here.
Connes wrote in 2001 as follows:
What conclusion can one draw about nonstandard analysis? This means that, since noone will ever be able to name a nonstandard number, the theory remains virtual (Connes et al. 2001, p. 16)
The exact meaning of the verb "to name" used by Connes here is not entirely clear. Connes provided a hint as to its meaning in 2000, in the following terms:
if you are given a non standard number you can canonically produce a subset of the interval which is not Lebesgue measurable. Now we know from logic (from results of Paul Cohen and Solovay) that it will forever be impossible to produce explicitely [sic] a subset of the real numbers, of the interval [0, 1], say, that is not Lebesgue measurable (Connes 2000a, p. 21, 2004, p. 14).
The reference to Solovay indicates that Connes is relying on the result, which may be found in Solovay (1970, p. 3, Theorem 2) on the existence of a model S of set theory ZFC, in which (it is true that) every set of reals definable from a countable sequence of ordinals is Lebesgue measurable. Thus when Connes claims that the theory remains virtual he apparently means that the theory remains undefinable.
This however is contradicted by the existence of a hyperreal field definable in ZF, with transfer probable using only the countable axiom of choice, and a further mild well-orderability assumption (weaker than well-orderability of the reals taken for granted by Connes) ensuring properness; see this answer for details.
A Google search turns up a page on nonstandard analysis on WorldLingo that cites a quote from Connes in the section on criticisms. This may give you more understanding of what Connes is referring to, given that this seems to be a re-statement of what you have quoted above:
The answer given by nonstandard analysis, a so-called nonstandard real, is equally deceiving. From every nonstandard real number one can construct canonically a subset of the interval [0, 1], which is not Lebesgue measurable. No such set can be exhibited (Stern, 1985). This implies that not a single nonstandard real number can actually be exhibited.
The next remark is:
A. Connes Noncommutative Geometry and Space-Time, Page 55 in The Geometric Universe, Huggett et al. The point of Connes' criticism is that nonstandard hyperreals are as fictitious as non-measurable sets. These sets can be shown to exist, assuming the axiom of choice of set theory, but are not constructible. Non-measurable sets are usually considered pathological, a sort of irritant that must be tolerated in order to have the axiom of choice available.
As there are sources listed, you may be able to get some additional insight in reading some of the references.
http://www.worldlingo.com/ma/enwiki/en/Non-standard_analysis#Criticisms