Union with anonymous struct with flexible array member

The second case fails to compile, because flexible array member is a property of a structure type, not for unions. That's straightforward.

Next, in the first case, trying to access b[0] would be undefined behavior, as no memory has been allocated for that.

Quoting C11, §6.7.2.1/P18

As a special case, the last element of a structure with more than one named member may have an incomplete array type; this is called a flexible array member. [...] If this array would have no elements, it behaves as if it had one element but the behavior is undefined if any attempt is made to access that element or to generate a pointer one past it.

That said,

The members of an anonymous structure or union are considered to be members of the containing structure or union.

That is for the access purpose, the layout remains unchanged. See, in your first example, you're accessing a (and b) as if they are direct members of the union.

To clarify,

#include <stdio.h>

union test{
    struct {
        int p;
        float q;
    } t;                //named structure member
  struct {
      int a;
      int b[];
  };
    char pqr;
};


int main(void){
    union test test;
    test.t.p = 20;   // you have to use the structure member name to access the elements
    test.pqr = 'c';     // direct access, as member of union
    test.a = 10;        // member of anonymous structure, so it behaves as if direct member of union
}

The (C11) standard says in §6.7.2.1 Structure and union specifiers ¶3 — a constraint:

¶3 A structure or union shall not contain a member with incomplete or function type (hence, a structure shall not contain an instance of itself, but may contain a pointer to an instance of itself), except that the last member of a structure with more than one named member may have incomplete array type; such a structure (and any union containing, possibly recursively, a member that is such a structure) shall not be a member of a structure or an element of an array.

Note that only structures can (directly) contain a flexible array member — unions cannot.

The first case is legitimate. The second is not.

(It's §6.7.2.1 ¶18 that defines the term flexible array member.)

Incidentally, in the first version of the question, the printf() statement in the first example was accessing an element of the array that was not allocated — a defect that has since been fixed in revision 2. Writing union test test; gives you an array of size 0. You must use dynamic memory allocation (or some other mechanism) to allocate a union or structure with sufficient space for a non-empty FAM. Similar comments applied to the second example too.

Some Name says in a comment

But since in the first case the structure is an anonymous so the members of the structure should be considered as members of the containing union making the union to contain a flexible array member. As I quoted The members of an anonymous structure or union are considered to be members of the containing structure or union.

Note that the anonymous structure doesn't lose its shape just because it is embedded into a union. One difference is that the offset of b in union test cannot be 0 — which is completely different from normal members of a union. Normally, all members of a union start at offset 0. Mostly, though, that says that given a variable union test u;, you can refer to u.a and u.b. In times past, you would have had to specify a name for the structure: union test { struct { int a; int b[]; } s; }; and have used u.s.a or u.s.b to access the elements of the structure within the union. That doesn't affect where a flexible array member is allowed; only the notation used to access it.

It was certainly always the intention that untagged composites in an anonymous composite retain their shape. But that was not explicit in the wording of §6.7.2.1p13. The wording was revised tin C18 to: (emphasis added):

13. An unnamed member whose type specifier is a structure specifier with no tag is called an anonymous structure; an unnamed member whose type specifier is a union specifier with no tag is called an anonymous union. The members of an anonymous structure or union are considered to be members of the containing structure or union, keeping their structure or union layout. This applies recursively if the containing structure or union is also anonymous.

See http://www.iso-9899.info/wiki/The_Standard for links to the C18 standard and freely-available draft (pdf)

NOTE: this answer has been substantively modified since first being written, reflecting a change to the committee's position after publication of the documents on which the original version of the answer relied.

The members of an anonymous structure or union are considered to be members of the containing structure or union.

This is a tricky provision to interpret, and indeed it has been the subject of at least two defect reports against the standard. The intention, as supported by the committee in its response to DR 499 is that anonymous structures are treated for layout purposes as if the structure itself were the member of the containing structure or union, but access to its members is expressed as if they were members of the containing structure or union.

The accepted position on DR 502, on the other hand, holds that even an anonymous struct containing a flexible array member as its only member is allowed if it is the last member of the structure (not union) containing it, and at least one other precedes it.

I find those a bit inconsistent, but the unifying theme across them seems to be that the intent of the standard in this area comes down to layout. A flexible array member inside an anonymous struct is allowed as long as it comes at the end of the layout of the innermost named structure or union, which must have non-zero size from consideration of the other members, taking into consideration the fact that members of an anonymous struct do not overlap, regardless of whether the anonymous struct appears inside a union.

The proposed committee response to DR 502 (which differs from its initial position) is consistent with that. It holds that anonymous structures inside a structure or union must obey the same rules as other structures with respect to flexible array members, notwithstanding the provision you ask about.

The committee does not appear to have decided the specific question you asked, but the theme of its decisions seems clear: the "considered to be members of the containing structure or union" wording is intended to be interpreted narrowly, as a statement only about the syntax for accessing members of anonymous structures and unions. Thus, that provision has nothing to say about whether anonymous structures may contain FAMs, and the general rules about when and where they may do apply. Those rules allow your first case.