The values for icons are located in registry.
Go to RUN and type regedit
Go to HKEY_CLASSES_ROOT\.html
Then check the default. For example I have chromium browser So my default key is ChromiumHTM.
Go to the name of key in default:
HKEY_CLASSES_ROOT\\DefaultIcon
Example:
HKEY_CLASSES_ROOT\ChromiumHTM\DefaultIcon
Change the default value for that key to desired icon. Restart explorer.exe and your icon is changed.