Worked it out!
My .ssh\config
file was referencing the identity file (ssh private key) with a home-path, I changed this to an absolute path and both console and tortoise-git both started working :)
IdentityFile ~/.ssh/id_rsa
changed to
IdentityFile C:\Users\user.name\.ssh\id_rsa