gcc/clang usage of restrict keyword for local variables and struct fields

restrict is not defined by the C standard for members of structures.

The formal definition of restrict in 6.7.3.1 begins with “Let D be a declaration of an ordinary identifier…”

6.2.3 1 defines defines “ordinary identifiers“ to exclude members of structures or unions:

… Thus, there are separate name spaces for various categories of identifiers, as follows:

— label names (disambiguated by the syntax of the label declaration and use);

— the tags of structures, unions, and enumerations (disambiguated by following any of the keywords struct, union, or enum);

— the members of structures or unions; each structure or union has a separate name space for its members (disambiguated by the type of the expression used to access the member via the . or -> operator);

— all other identifiers, called ordinary identifiers (declared in ordinary declarators or as enumeration constants).

Footnote 126 in 6.7.2.1 explicitly tells us structure members are not ordinary identifiers:

A structure or union cannot contain a member with a variably modified type because member names are not ordinary identifiers as defined in 6.2.3.

for what I see, it looks like restrict is only honored if specified in function arguments

That characterization sounds like you think restrict qualification carries some kind of obligation to optimize more aggressively. That explicitly is not the case:

A translator is free to ignore any or all aliasing implications of uses of restrict.

(C standard, paragraph 6.7.3.1/6)

I do grant that it is a bit surprising that a compiler that takes advantage of restrict qualification to perform additional optimizations in some cases would not do the same in other, similar cases, but that does not imply that either the code or the compiler is in any way wrong. (But do bear in mind Eric's observation about restrict-qualified structure members). Also, however, the examples presented may not all be as similar to each other as you suppose.

Is my usage of restrict wrong, or gcc/clang only implements restrict for function arguments and nothing else?

Although the standard defines semantics for restrict-qualified block-scoped variables, they cannot really be used for much. Restrict qualification is a means to move some responsibility for dependency analysis from compiler to programmer, but the programmer has no more information to bring to bear than the compiler already has in a case such as the foo_restricted_cast() example. I'd say that yes, your usage there is (semantically) wrong, because you have no sound basis on which to make the implicit guarantee that local variables a and b will not alias each other. I rate GCC's and Clang's behavior prudent and appropriate in that light, and ICC's somewhat rash.

As for restrict-qualified structure members, I disagree with the other answer's assertion that no semantics are defined for them. It is true that the identifiers of structure members are not "ordinary identifiers", but the wording of the standard's definition of restrict semantics seems to be specifically crafted with a view toward covering structure members via the declarations of the ordinary identifiers of structure objects containing them. The language can certainly be read that way, and it is more than usually fraught if it is meant otherwise.

Thus, I think that the case of foo_restricted_struct() has well-defined semantics, and furthermore that icc is justified in taking advantage of the non-aliasing assertions conveyed by the restrict qualification of the argument structure's members, just as if they were direct function parameters. It is impossible for me to say why gcc and Clang do not also take advantage of the optimization options that proceed, but again, they have no obligation to do so.

On the other hand, foo_restricted_subcall() exhibits a semantic problem similar to the one in foo_restricted_cast(). I suppose that there is an outside chance that it is for that reason that GCC and/or Clang avoids more aggressively optimizing foo_restricted_struct(), which foo_restricted_subcall() calls with a semantically problematic argument. Probably, though, these compilers just don't perform a deep enough analysis to see the optimization opportunity in this case.