Set standard file browser for "open containing folder"

You can define the default file browser by editing the file ~/.local/share/applications/mimeapps.list. Open this file and change the line inode/directory as follow

inode/directory=nautilus.desktop;

If this doesn't work, you should change the filemanager in the file

/usr/share/applications/mimeinfo.cache

by adding (or updating) this line

inode/directory=nautilus.desktop

The following command solved this problem in arch linux:

xdg-mime default nautilus.desktop inode/directory

(Source: Arch Forums)

You can find a comprehensive description of setting up default applications here in the according Arch wiki article.

For your specific case, you edit ~/.config/mimeapps.list and add a new line with inode/directory=nautilus.desktop to the section [Default Applications].

The file should look similar to the following one:

[Default Applications]
text/plain=SciTE.desktop 
text/x-tex=texmaker.desktop 
inode/directory=nautilus.desktop