Set custom folder icon for a network folder in Windows File Explorer

You'll want to use a desktop.ini file to customize the icon. Usually, that's all your actually doing when you are using that "Change Icon" button above. For full documentation see: How to customize folders with desktop.ini An example desktop.ini file follows:

[DeleteOnCopy]
Owner=MyUser
Personalized=13
PersonalizedName=My Folder
[.ShellClassInfo]
[email protected],-12689
IconFile=%SystemRoot%\system32\shell32.dll
IconIndex=-237

As an aside, if you want to do this without having to manually create the desktop.ini, you can create a folder locally, apply the desired settings, and steal its desktop.ini. (If you don't see it, copy over the entire folder; it should have a correct name originally.) The youtube video Apply a Icon to a Folder on a Network Drive shows this process but uses folder moves rather than creating a new folder.

The customization method outlined by ssnobody assumes you have the capability to change the folder on the remote machine, and that you want the appearance to be changed for all remote and local users of the folder. If either of these is not true, you might want to use this alternative: create a local shortcut to the remote folder (e.g. right-click drag it to the local desktop and select "create shortcut"), and then use the normal "change icon" button of the properties dialog to change the shortcut's icon.