Ok so this is how I resolved it. I downloaded the Windows 8.1 recovery using Media Creation Tool. It downloads some files and you need to find "install.esd" under sources folder and convert it to "install.wim" using conversion tools like NtLite,Winreducer etc. I followed this Article to mount the converted "install.wim" and then copy "winRE.wim" from mountedimage\windows\system32\recovery to my my system's C:\Windows\System32\Recovery folder to restore the Reset/Refresh mechanism of Windows and then copied "install.wim" on C:\ and pointed the Windows to this location to look for any files required for Reset/Refresh (as explained in the article)
After this I was able to refresh my system back to Windows 8.1 Single language.:)