MakeExpression can't deal with FE boxes riffled with whitespaces
Tracing the evaluation with Trace[ToExpression[boxes], TraceInternal -> True]
reveals that ToExpression
calls StripBoxes
as the first step which removes the redundant spaces. So the obvious workaround is
MakeExpression[StripBoxes@boxes, StandardForm]
which seems to work.
Note however that according to the Docs page for StripBoxes
,
When boxes are sent by the front end to the kernel for evaluation, the front end strips the boxes using a more extensive process that may depend upon active stylesheets and front end option settings.
This is not a bug. MakeExpression
, as a low-level function, only deals with canonical boxes as sent by the FE. Sometimes you get lucky with non-canonical boxes, but there's no guarantee. ToExpression
, which is a more user-friendly function, will apply StripBoxes
first, which attempts to simulate the canonicalization performed by the FE, so it will work on more cases (except for the occasional bug in StripBoxes
, in which case you might end up with wrong thing).
The documentation example referenced in the comments was added in 12.1, but this has been the intent forever. It is perhaps still underdocumented.
BTW, newer versions of the kernel will attempt to ask the FE, if it is available, to resolve template boxes--and make their best guess from Core.nb if it is not available. Hence, John's statement about only the FE knowing the how to resolve some boxes, while still true, is not as relevant as it used to be.