The answer to your question is to Opt + Cmd + Click on the point you want to move. This will allow you to move the point without activating the selection handle.
Found on https://support.omnigroup.com/documentation/en/omnigraffle-6-for-mac/using-the-tools