Singleton pattern (Bill Pugh's solution)

I think Mr Pugh's version is held in high regard because it only performs the instantiation of the singleton when getInstance() is called i.e. not when the class (the class holding the getInstance method) is loaded. If your singleton construction does something costly then this may be an advantage for you. If you're like the majority of the world whose singletons are just to avoid static methods (and you haven't moved onto dependency injection frameworks), then I would not lose any sleep over it.

As the article states, Mr Pugh's method is lazier than the static instance variable - but in reality if the Singleton class gets loaded you're going to be calling the getInstance method anyhow. So as a computer science exercise it's useful, but in the real world its benefits are debatable.

p.s. I don't care much for Mr Bloch's example here as to use an enum would be to say My Singleton IS-A enum, which doesn't sound right to me (especially from someone who, rightly, says never implement an interface just to get the constants)

The JLS guarantees that a class is only loaded when it's used for the first time (making the singleton initialization lazy), and that the class loading is thread-safe (making the getInstance() method thread-safe as well)

As for why thread-safe

Because the first time getInstance() is called, the JVM will hold the holder class. If another thread calls getInstance() concurrently, the JVM won't load the holder class a second time: it will wait for the first thread to have completed the class loading, and at the end of the loading and initialization of the holder class, both thread will see the holder class properly initialized and thus containing the unique singleton instance.

Is it because a static class is not load by the VM before it's actually used

Not just a static class, any class. Classes aren't loaded until they are referenced. See the JLS - 12.4.1 When Initialization Occurs

or something like this, so we don't create the object before we turn to the getInstance() method?

Exactly.

Also is that method thread safe only to the extent of initializing the object?

Handing out a reference is thread-safe, so this method is always thread-safe, not just at creation time