Why does $S = k_B \ln W$ not always apply?

The problem with the Boltzmann definition is, as you have neatly shown, that its usefulness depends on the assumption that your system is in equilibrium with its surroundings. Without first assuming equilibrium and subsequently setting the temperatures as equal, one cannot show that the Boltzmann entropy satisfies the First Law and hence meaningfully define it as the entropy. However the Gibbs entropy still does provide a meaningful definition because, for example, it is possible to relate it to the partition function via

$$S \equiv -k_B\sum_s P_s\ln P_s= k_B \left(\ln Z + \beta\frac{\partial \ln Z}{\partial \beta}\right)=\frac{\partial}{\partial T}(k_B T \ln Z)$$

and hence to use it to calculate other variables like the Helmholtz free energy.

Furthermore, you are assuming that $W$ is large enough to be approximated as a continuous quantity. $W$ is however an integer and the quantity $\mathrm d\ln W$ is not well defined in a system with a small number of microstates. Hence the justification you have provided here would break down. This problem never arises if you start from the definition of the Gibbs entropy and work from there.