When does invoking a member function on a null instance result in undefined behavior?
Both (a) and (b) result in undefined behavior. It's always undefined behavior to call a member function through a null pointer. If the function is static, it's technically undefined as well, but there's some dispute.
The first thing to understand is why it's undefined behavior to dereference a null pointer. In C++03, there's actually a bit of ambiguity here.
Although "dereferencing a null pointer results in undefined behavior" is mentioned in notes in both §1.9/4 and §8.3.2/4, it's never explicitly stated. (Notes are non-normative.)
However, one can try to deduced it from §3.10/2:
An lvalue refers to an object or function.
When dereferencing, the result is an lvalue. A null pointer does not refer to an object, therefore when we use the lvalue we have undefined behavior. The problem is that the previous sentence is never stated, so what does it mean to "use" the lvalue? Just even generate it at all, or to use it in the more formal sense of perform lvalue-to-rvalue conversion?
Regardless, it definitely cannot be converted to an rvalue (§4.1/1):
If the object to which the lvalue refers is not an object of type T and is not an object of a type derived from T, or if the object is uninitialized, a program that necessitates this conversion has undefined behavior.
Here it's definitely undefined behavior.
The ambiguity comes from whether or not it's undefined behavior to deference but not use the value from an invalid pointer (that is, get an lvalue but not convert it to an rvalue). If not, then int *i = 0; *i; &(*i); is well-defined. This is an active issue.
So we have a strict "dereference a null pointer, get undefined behavior" view and a weak "use a dereferenced null pointer, get undefined behavior" view.
Now we consider the question.
Yes, (a) results in undefined behavior. In fact, if this is null then regardless of the contents of the function the result is undefined.
This follows from §5.2.5/3:
If
E1has the type “pointer to class X,” then the expressionE1->E2is converted to the equivalent form(*(E1)).E2;
*(E1) will result in undefined behavior with a strict interpretation, and .E2 converts it to an rvalue, making it undefined behavior for the weak interpretation.
It also follows that it's undefined behavior directly from (§9.3.1/1):
If a nonstatic member function of a class X is called for an object that is not of type X, or of a type derived from X, the behavior is undefined.
With static functions, the strict versus weak interpretation makes the difference. Strictly speaking, it is undefined:
A static member may be referred to using the class member access syntax, in which case the object-expression is evaluated.
That is, it's evaluated just as if it were non-static and we once again dereference a null pointer with (*(E1)).E2.
However, because E1 is not used in a static member-function call, if we use the weak interpretation the call is well-defined. *(E1) results in an lvalue, the static function is resolved, *(E1) is discarded, and the function is called. There is no lvalue-to-rvalue conversion, so there's no undefined behavior.
In C++0x, as of n3126, the ambiguity remains. For now, be safe: use the strict interpretation.
Obviously undefined means it's not defined, but sometimes it can be predictable. The information I'm about to provide should never be relied on for working code since it certainly isn't guaranteed, but it might come in useful when debugging.
You might think that calling a function on an object pointer will dereference the pointer and cause UB. In practice if the function isn't virtual, the compiler will have converted it to a plain function call passing the pointer as the first parameter this, bypassing the dereference and creating a time bomb for the called member function. If the member function doesn't reference any member variables or virtual functions, it might actually succeed without error. Remember that succeeding falls within the universe of "undefined"!
Microsoft's MFC function GetSafeHwnd actually relies on this behavior. I don't know what they were smoking.
If you're calling a virtual function, the pointer must be dereferenced to get to the vtable, and for sure you're going to get UB (probably a crash but remember that there are no guarantees).