Real-world use of X-Macros

Some real-world uses of X-Macros by popular and large projects:

Java HotSpot

In the Oracle HotSpot Virtual Machine for the Java® Programming Language, there is the file globals.hpp, which uses the RUNTIME_FLAGS in that way.

See the source code:

• JDK 7
• JDK 8
• JDK 9

Chromium

The list of network errors in net_error_list.h is a long, long list of macro expansions of this form:

NET_ERROR(IO_PENDING, -1)

It is used by net_errors.h from the same directory:

enum Error {
  OK = 0,

#define NET_ERROR(label, value) ERR_ ## label = value,
#include "net/base/net_error_list.h"
#undef NET_ERROR
};

The result of this preprocessor magic is:

enum Error {
  OK = 0,
  ERR_IO_PENDING = -1,
};

What I don't like about this particular use is that the name of the constant is created dynamically by adding the ERR_. In this example, NET_ERROR(IO_PENDING, -100) defines the constant ERR_IO_PENDING.

Using a simple text search for ERR_IO_PENDING, it is not possible to see where this constant it defined. Instead, to find the definition, one has to search for IO_PENDING. This makes the code hard to navigate and therefore adds to the obfuscation of the whole code base.

X-Macros are essentially parameterized templates. So they are the right tool for the job if you need several similar things in several guises. They allow you to create an abstract form and instantiate it according to different rules.

I use X-macros to output enum values as strings. And since encountering it, I strongly prefer this form which takes a "user" macro to apply to each element. Multiple file inclusion is just far more painful to work with.

/* x-macro constructors for error and type
   enums and string tables */
#define AS_BARE(a) a ,
#define AS_STR(a) #a ,

#define ERRORS(_) \
    _(noerror) \
    _(dictfull) _(dictstackoverflow) _(dictstackunderflow) \
    _(execstackoverflow) _(execstackunderflow) _(limitcheck) \
    _(VMerror)
enum err { ERRORS(AS_BARE) };
char *errorname[] = { ERRORS(AS_STR) };
/* puts(errorname[(enum err)limitcheck]); */

I'm also using them for function dispatch based on object type. Again by hijacking the same macro I used to create the enum values.

#define TYPES(_) \
    _(invalid) \
    _(null) \
    _(mark) \
    _(integer) \
    _(real) \
    _(array) \
    _(dict) \
    _(save) \
    _(name) \
    _(string) \
/*enddef TYPES */

#define AS_TYPE(_) _ ## type ,
enum { TYPES(AS_TYPE) };

Using the macro guarantees that all my array indices will match the associated enum values, because they construct their various forms using the bare tokens from the macro definition (the TYPES macro).

typedef void evalfunc(context *ctx);

void evalquit(context *ctx) { ++ctx->quit; }

void evalpop(context *ctx) { (void)pop(ctx->lo, adrent(ctx->lo, OS)); }

void evalpush(context *ctx) {
    push(ctx->lo, adrent(ctx->lo, OS),
            pop(ctx->lo, adrent(ctx->lo, ES)));
}

evalfunc *evalinvalid = evalquit;
evalfunc *evalmark = evalpop;
evalfunc *evalnull = evalpop;
evalfunc *evalinteger = evalpush;
evalfunc *evalreal = evalpush;
evalfunc *evalsave = evalpush;
evalfunc *evaldict = evalpush;
evalfunc *evalstring = evalpush;
evalfunc *evalname = evalpush;

evalfunc *evaltype[stringtype/*last type in enum*/+1];
#define AS_EVALINIT(_) evaltype[_ ## type] = eval ## _ ;
void initevaltype(void) {
    TYPES(AS_EVALINIT)
}

void eval(context *ctx) {
    unsigned ades = adrent(ctx->lo, ES);
    object t = top(ctx->lo, ades, 0);
    if ( isx(t) ) /* if executable */
        evaltype[type(t)](ctx);  /* <--- the payoff is this line here! */
    else
        evalpush(ctx);
}

Using X-macros this way actually helps the compiler to give helpful error messages. I omitted the evalarray function from the above because it would distract from my point. But if you attempt to compile the above code (commenting-out the other function calls, and providing a dummy typedef for context, of course), the compiler would complain about a missing function. For each new type I add, I am reminded to add a handler when I recompile this module. So the X-macro helps to guarantee that parallel structures remain intact even as the project grows.

Edit:

This answer has raised my reputation 50%. So here's a little more. The following is a negative example, answering the question: when not to use X-Macros?

This example shows the packing of arbitrary code fragments into the X-"record". I eventually abandoned this branch of the project and did not use this strategy in later designs (and not for want of trying). It became unweildy, somehow. Indeed the macro is named X6 because at one point there were 6 arguments, but I got tired of changing the macro name.

/* Object types */
/* "'X'" macros for Object type definitions, declarations and initializers */
// a                      b            c              d
// enum,                  string,      union member,  printf d
#define OBJECT_TYPES \
X6(    nulltype,        "null",     int dummy      ,            ("<null>")) \
X6(    marktype,        "mark",     int dummy2      ,           ("<mark>")) \
X6( integertype,     "integer",     int  i,     ("%d",o.i)) \
X6( booleantype,     "boolean",     bool b,     (o.b?"true":"false")) \
X6(    realtype,        "real",     float f,        ("%f",o.f)) \
X6(    nametype,        "name",     int  n,     ("%s%s", \
        (o.flags & Fxflag)?"":"/", names[o.n])) \
X6(  stringtype,      "string",     char *s,        ("%s",o.s)) \
X6(    filetype,        "file",     FILE *file,     ("<file %p>",(void *)o.file)) \
X6(   arraytype,       "array",     Object *a,      ("<array %u>",o.length)) \
X6(    dicttype,        "dict",     struct s_pair *d, ("<dict %u>",o.length)) \
X6(operatortype,    "operator",     void (*o)(),    ("<op>")) \

#define X6(a, b, c, d) #a,
char *typestring[] = { OBJECT_TYPES };
#undef X6

// the Object type
//forward reference so s_object can contain s_objects
typedef struct s_object Object;

// the s_object structure:
// a bit convoluted, but it boils down to four members:
// type, flags, length, and payload (union of type-specific data)
// the first named union member is integer, so a simple literal object
// can be created on the fly:
// Object o = {integertype,0,0,4028}; //create an int object, value: 4028
// Object nl = {nulltype,0,0,0};
struct s_object {
#define X6(a, b, c, d) a,
    enum e_type { OBJECT_TYPES } type;
#undef X6
unsigned int flags;
#define Fread  1
#define Fwrite 2
#define Fexec  4
#define Fxflag 8
size_t length; //for lint, was: unsigned int
#define X6(a, b, c, d) c;
    union { OBJECT_TYPES };
#undef X6
};

One big problem was the printf format strings. While it looks cool, it's just hocus pocus. Since it's only used in one function, overuse of the macro actually separated information that should be together; and it makes the function unreadable by itself. The obfuscation is doubly unfortunate in a debugging function like this one.

//print the object using the type's format specifier from the macro
//used by O_equal (ps: =) and O_equalequal (ps: ==)
void printobject(Object o) {
    switch (o.type) {
#define X6(a, b, c, d) \
        case a: printf d; break;
OBJECT_TYPES
#undef X6
    }
}

So don't get carried away. Like I did.

I discovered X-macros a couple of years ago when I started making use of function pointers in my code. I am an embedded programmer and I use state machines frequently. Often I would write code like this:

/* declare an enumeration of state codes */
enum{ STATE0, STATE1, STATE2, ... , STATEX, NUM_STATES};

/* declare a table of function pointers */
p_func_t jumptable[NUM_STATES] = {func0, func1, func2, ... , funcX};

The problem was that I considered it very error prone to have to maintain the ordering of my function pointer table such that it matched the ordering of my enumeration of states.

A friend of mine introduced me to X-macros and it was like a light-bulb went off in my head. Seriously, where have you been all my life x-macros!

So now I define the following table:

#define STATE_TABLE \
        ENTRY(STATE0, func0) \
        ENTRY(STATE1, func1) \
        ENTRY(STATE2, func2) \
        ...
        ENTRY(STATEX, funcX) \

And I can use it as follows:

enum
{
#define ENTRY(a,b) a,
    STATE_TABLE
#undef ENTRY
    NUM_STATES
};

and

p_func_t jumptable[NUM_STATES] =
{
#define ENTRY(a,b) b,
    STATE_TABLE
#undef ENTRY
};

as a bonus, I can also have the pre-processor build my function prototypes as follows:

#define ENTRY(a,b) static void b(void);
    STATE_TABLE
#undef ENTRY

Another usage is to declare and initialize registers

#define IO_ADDRESS_OFFSET (0x8000)
#define REGISTER_TABLE\
    ENTRY(reg0, IO_ADDRESS_OFFSET + 0, 0x11)\
    ENTRY(reg1, IO_ADDRESS_OFFSET + 1, 0x55)\
    ENTRY(reg2, IO_ADDRESS_OFFSET + 2, 0x1b)\
    ...
    ENTRY(regX, IO_ADDRESS_OFFSET + X, 0x33)\

/* declare the registers (where _at_ is a compiler specific directive) */
#define ENTRY(a, b, c) volatile uint8_t a _at_ b:
    REGISTER_TABLE
#undef ENTRY

/* initialize registers */
#define ENTRY(a, b, c) a = c;
    REGISTER_TABLE
#undef ENTRY

My favourite usage however is when it comes to communication handlers

First I create a comms table, containing each command name and code:

#define COMMAND_TABLE \
    ENTRY(RESERVED,    reserved,    0x00) \
    ENTRY(COMMAND1,    command1,    0x01) \
    ENTRY(COMMAND2,    command2,    0x02) \
    ...
    ENTRY(COMMANDX,    commandX,    0x0X) \

I have both the uppercase and lowercase names in the table, because the upper case will be used for enums and the lowercase for function names.

Then I also define structs for each command to define what each command looks like:

typedef struct {...}command1_cmd_t;
typedef struct {...}command2_cmd_t;

etc.

Likewise I define structs for each command response:

typedef struct {...}command1_resp_t;
typedef struct {...}command2_resp_t;

etc.

Then I can define my command code enumeration:

enum
{
#define ENTRY(a,b,c) a##_CMD = c,
    COMMAND_TABLE
#undef ENTRY
};

I can define my command length enumeration:

enum
{
#define ENTRY(a,b,c) a##_CMD_LENGTH = sizeof(b##_cmd_t);
    COMMAND_TABLE
#undef ENTRY
};

I can define my response length enumeration:

enum
{
#define ENTRY(a,b,c) a##_RESP_LENGTH = sizeof(b##_resp_t);
    COMMAND_TABLE
#undef ENTRY
};

I can determine how many commands there are as follows:

typedef struct
{
#define ENTRY(a,b,c) uint8_t b;
    COMMAND_TABLE
#undef ENTRY
} offset_struct_t;

#define NUMBER_OF_COMMANDS sizeof(offset_struct_t)

NOTE: I never actually instantiate the offset_struct_t, I just use it as a way for the compiler to generate for me my number of commands definition.

Note then I can generate my table of function pointers as follows:

p_func_t jump_table[NUMBER_OF_COMMANDS] = 
{
#define ENTRY(a,b,c) process_##b,
    COMMAND_TABLE
#undef ENTRY
}

And my function prototypes:

#define ENTRY(a,b,c) void process_##b(void);
    COMMAND_TABLE
#undef ENTRY

Now lastly for the coolest use ever, I can have the compiler calculate how big my transmit buffer should be.

/* reminder the sizeof a union is the size of its largest member */
typedef union
{
#define ENTRY(a,b,c) uint8_t b##_buf[sizeof(b##_cmd_t)];
    COMMAND_TABLE
#undef ENTRY
}tx_buf_t

Again this union is like my offset struct, it is not instantiated, instead I can use the sizeof operator to declare my transmit buffer size.

uint8_t tx_buf[sizeof(tx_buf_t)];

Now my transmit buffer tx_buf is the optimal size and as I add commands to this comms handler, my buffer will always be the optimal size. Cool!

One other use is to create offset tables: Since memory is often a constraint on embedded systems, I don't want to use 512 bytes for my jump table (2 bytes per pointer X 256 possible commands) when it is a sparse array. Instead I will have a table of 8bit offsets for each possible command. This offset is then used to index into my actual jump table which now only needs to be NUM_COMMANDS * sizeof(pointer). In my case with 10 commands defined. My jump table is 20bytes long and I have an offset table that is 256 bytes long, which is a total of 276bytes instead of 512bytes. I then call my functions like so:

jump_table[offset_table[command]]();

instead of

jump_table[command]();

I can create an offset table like so:

/* initialize every offset to 0 */
static uint8_t offset_table[256] = {0};

/* for each valid command, initialize the corresponding offset */
#define ENTRY(a,b,c) offset_table[c] = offsetof(offset_struct_t, b);
    COMMAND_TABLE
#undef ENTRY

where offsetof is a standard library macro defined in "stddef.h"

As a side benefit, there is a very easy way to determine if a command code is supported or not:

bool command_is_valid(uint8_t command)
{
    /* return false if not valid, or true (non 0) if valid */
    return offset_table[command];
}

This is also why in my COMMAND_TABLE I reserved command byte 0. I can create one function called "process_reserved()" which will be called if any invalid command byte is used to index into my offset table.