How is thermodynamic entropy defined? What is its relationship to information entropy?

I think that the best way to justify the logarithm is that you want entropy to be an extensive quantity -- that is, if you have two non-interacting systems A and B, you want the entropy of the combined system to be $$ S_{AB}=S_A+S_B. $$ If the two systems have $N_A,N_B$ states each, then the combined system has $N_AN_B$ states. So to get additivity in the entropy, you need to take the log.

You might wonder why it's so important that the entropy be extensive (i.e., additive). That's partly just history. Before people had worked out the microscopic basis for entropy, they'd worked out a lot of the theory on macroscopic thermodynamic grounds alone, and the quantity that they'd defined as entropy was additive.

Also, the number of states available to a macroscopic system tends to be absurdly, exponentially large, so if you don't take logarithms it's very inconvenient: who wants to be constantly dealing with numbers like $10^{10^{20}}$?

The thermodynamic entropy is not what you wrote. That is the information entropy. Microstates and the counting of are not thermodynamic concepts, but rather statistical mechanics ones. Thermodynamics proceeds by identifying reversible processes, and goes on to study them. A key point is noticing that there are quantities called state variables which are intrinsic to a substance, which do not depend on the path used to obtain it, such as temperature, or pressure. Heat transfer is explicitly (and experimentally) not one such quantity. However, by considering a perfect reversible heat engine, it is possible to show that the quantity $dQ/T$ is a state variable. Integrating it gives $$\Delta S = \int \frac{dQ}{T}$$ and is the change of entropy. Thermodynamically one can only talk of changes, and the zero is not particularly well defined --- the 3rd law of thermodynamics tries to define it, but it is not always applicable.

The connection between thermodynamic and information theoretic entropies is a deep connection, and has been an active research area for more than a century.

To get more information, you can look at

• This well written comprehensive review at Stanford Encyclopedia of Philosophy, including the link with macroscopic thermodynamical entropy.
• This webpage and this wikipedia page on the subject give shorter overviews of some of the links between these entropies.

Various Researches on Maxwell's demon,Szilard's engine and Landauer's principle can also be helpful.