Disable only Excel previews in Windows 10 folder preview pane
There is a "Preview Handler" associated with every file extension in the registry, that can be disabled.
The known file extensions are under HKEY_CLASSES_ROOT as ".xlsx" for example. If the extension has a Preview Handler, then there is a subkey always called "HKEY_CLASSES_ROOT\.xlsx\ShellEx\{8895b1c6-b41f-4c1c-a562-0d564250836f}". That subkey has a default value that points to the Preview Handler, if you clear that default value or remove the subkey, the preview for that file extension will be disabled. You could store a copy of that key by going right-click -> Export, so that you can restore the preview handler later if need be.
I gathered this from a Microsoft document here, even though they describe an extra "step" in mapping the preview handler, which i found not to be the case on Windows 7, 10 and Server 2016. Maybe someone can explain/correct this.