How is `>>>` lexed in C++0x?
The text of the FDIS says
Similarly, the first non-nested >> is treated as two consecutive but distinct > tokens
It cannot unlex tokens and relex. So this will be a > > >
. Note that the input to a C++ implementation is first lexed into preprocessing tokens, and then those tokens are converted into C++ tokens. So first your input are the C++ tokens >> >
, then the C++ parser changes these to > > >
.
Each preprocessing token is converted into a token. (2.7). The resulting tokens are syntactically and semantically analyzed and translated as a translation unit. [ Note: The process of analyzing and translating the tokens may occasionally result in one token being replaced by a sequence of other tokens (14.2). — end note ]
There's no chance you could merge those two trailing > >
tokens.