C macro to create a bit mask -- possible? And have I found a GCC bug?

Here is a version of the macro which will work for arbitrary positive inputs. (Negative inputs still invoke undefined behavior...)

#include <limits.h>
/* A mask with x least-significant bits set, possibly 0 or >=32 */
#define BIT_MASK(x) \
    (((x) >= sizeof(unsigned) * CHAR_BIT) ?
        (unsigned) -1 : (1U << (x)) - 1)

Of course, this is a somewhat dangerous macro as it evaluates its argument twice. This is a good opportunity to use a static inline if you use GCC or target C99 in general.

static inline unsigned bit_mask(int x)
{
    return (x >= sizeof(unsigned) * CHAR_BIT) ?
        (unsigned) -1 : (1U << x) - 1;
}

As Mysticial noted, shifting more than 32 bits with a 32-bit integer results in implementation-defined undefined behavior. Here are three different implementations of shifting:

• On x86, only examine the low 5 bits of the shift amount, so x << 32 == x.
• On PowerPC, only examine the low 6 bits of the shift amount, so x << 32 == 0 but x << 64 == x.
• On Cell SPUs, examine all bits, so x << y == 0 for all y >= 32.

However, compilers are free to do whatever they want if you shift a 32-bit operand 32 bits or more, and they are even free to behave inconsistently (or make demons fly out your nose).

Implementing BIT_FIELD_MASK:

This will set bit a through bit b (inclusive), as long as 0 <= a <= 31 and 0 <= b <= 31.

#define BIT_MASK(a, b) (((unsigned) -1 >> (31 - (b))) & ~((1U << (a)) - 1))

#define BIT_MASK(foo) ((~ 0ULL) >> (64-foo))

I'm a bit paranoid about this. I think this assumes that unsigned long long is exactly 64 bits. But it's a start and it works up to 64 bits.

Maybe this is correct:

define BIT_MASK(foo) ((~ 0ULL) >> (sizeof(0ULL)*8-foo))

Shifting by more than or equal to the size of the integer type is undefined behavior.
So no, it's not a GCC bug.

In this case, the literal 1 is of type int which is 32-bits in both systems that you used. So shifting by 32 will invoke this undefined behavior.

---

In the first case, the compiler is not able to resolve the shift-amount to 32. So it likely just issues the normal shift-instruction. (which in x86 uses only the bottom 5-bits) So you get:

(unsigned int)(1 << 0) - 1

which is zero.

In the second case, GCC is able to resolve the shift-amount to 32. Since it is undefined behavior, it (apparently) just replaces the entire result with 0:

(unsigned int)(0) - 1

so you get ffffffff.

---

So this is a case of where GCC is using undefined behavior as an opportunity to optimize.
(Though personally, I'd prefer that it emits a warning instead.)

Related: Why does integer overflow on x86 with GCC cause an infinite loop?

Assuming you have a working mask for n bits, e.g.

// set the first n bits to 1, rest to 0
#define BITMASK1(n) ((1ULL << (n)) - 1ULL)

you can make a range bitmask by shifting again:

// set bits [k+1, n] to 1, rest to 0
#define BITNASK(n, k) ((BITMASK(n) >> k) << k)

The type of the result is unsigned long long int in any case.

As discussed, BITMASK1 is UB unless n is small. The general version requires a conditional and evaluates the argument twice:

#define BITMASK1(n) (((n) < sizeof(1ULL) * CHAR_BIT ? (1ULL << (n)) : 0) - 1ULL)