How do I show logarithmically spaced grid lines at all ticks on a log-log plot using Matplotlib?

Basically, you just need to put in the parameter which="both" in the grid command so that it becomes:

matplotlib.pyplot.grid(True, which="both")

Other options for which are 'minor' and 'major' which are the major ticks (which are shown in your graph) and the minor ticks which you are missing. If you want solid lines then you can use ls="-" as a parameter to grid() as well.

Here is an example for kicks:

import numpy as np
from matplotlib import pyplot as plt

x = np.arange(0, 100, .5)
y = 2 * x**3

plt.loglog(x, y)
plt.grid(True, which="both", ls="-")
plt.show()

which generates:

More details on the Matplotlib Docs

As @Bryce says, in older version of matplotlib correct kwarg is which=majorminor. I think that solid lines with a lighter color can be better than the dotted lines.

plt.grid(True, which="majorminor", ls="-", color='0.65')

Note that in the latest version of matplotlib this argument is replaced by 'both'.

plt.grid(True, which="both", ls="-", color='0.65')