I have managed to remove the orphaned contacts.
I uninstalled and reinstalled the People app (along with Mail, Calendar and Messaging apps, they are all in a single app)
- Right-Click on People tile
- Click
Uninstall
at the bottom of the screen - Click
Uninstall
to confirm removing Mail, Calendar, People and Messaging - Delete/Rename the folder
C:\Users\YOUR_USERNAME\AppData\Local\Packages\microsoft.windowscommunicationsapps_IDENTIFIER
- Reboot (possibly not required, but I did it anyway)
- Open the
Store
app - Hover in the bottom right corner and select
Search
from the menu which appears - Enter
Mail, Calendar, People and Messaging
in the search box - Select it from the
Recommendations
list - Click the
Install
button
I now have an empty People app!