How to change font size of the scientific notation in matplotlib?

I use ax.yaxis.set_major_formatter(ScalarFormatter(useMathText=True)) for changing 1e-4 to x10^-4

You want to change the fontsize of the offset_text from the yaxis. You can do that with this line:

ax.yaxis.get_offset_text().set_fontsize(24)

Or equivalently:

ax.yaxis.offsetText.set_fontsize(24)