Totally same as you, I'm a left handed artist drawing on the Surface Pro 4 and constantly activating the Task View swipe, it's pretty annoying.
After hours of looking on various forums to no avail, I finally found something that works! I downloaded "Skip Metro Suite," a third-party software, and just installed it and then rebooted. Hahaha I totally sound like a spam message right now, but it seems like it's working great so far (just installed it minutes ago before this post)! I can still use touchscreen, which is great (i.e. better than disabling the HID in Device Manager).
The forums I found this info in mentioned having to tweak some settings when using it for Surface Pro 3, but you have a Surface Pro 4 so I hope it works for you just as easily as it did for me!
I'm going on all the previous forums I visited and sharing this, haha. No Surface Pro 4 user should be without this tip! :D