Appears that Task Manager lies. Found someone with similar problem. Turns out of you drill down to Resource Monitor, the info can be totally different.
It was HPs SimplePass which has conflicts with Windows 10. Uninstalled SimplePass and the problem went away.
For Info: SimplePass is used for fingerprint scanning.