When to use addressof(x) instead of &x?

You use std::addressof when you have to. Sadly, "when you have to" includes anytime you are working in template code and want to turn a variable of unknown type T or T& into an honest-to-God pointer to that variable's memory.

Because the C++ committee foolishly allowed the overloading of the reference operator (to little legitimate purpose), it is possible for a user to instantiate your template with some type that you can't use the reference operator to get an actual pointer to. std::addressof is a way to work around users who use this dubious C++ feature in order to do what the language should have guaranteed to work to begin with.

In short, it's a library fix for a language stupidity. Use it in template code instead of & if you want to make sure users can't break your code. If your users can be trusted not to use this ill-conceived feature, then you can use &.

If it's a user-defined type with overloaded unary operator&, and you want its address, use addressof.

I'd say you should always use & because, as you say, if you don't, it defeats the purpose of the overload. Unless of course you do something meaningful with the overload, in which case you'd need addressof (outside the class, inside you can just use this), but you have to be very certain of what you're doing.

Here's more - if you want to overload operator& outside the class (you can), you have to use addressof to return the address, otherwise it results in infinite recursion:

struct Class
{
   virtual ~Class() {}
   int x;
};

void* operator&(const Class& x)
{
    //return &x; <---- infinite recursion
    return addressof(x) + 4; //I know this isn't safe
                             //but I also know the intrinsics of my compiler
                             //and platform to know this will actually return
                             //the address to the first data member
}

I know this isn't safe.

Use it when you want to know the actual address of the object, and not the result of an address-of operator& overload.