Removing any drive while defragmenting can cause damage to the file system. When you defragment a drive, the computer is constantly reading data from the drive and writing it to different parts of the drive. So if you were to remove the drive during this process, there is a strong possibility data that was read will not get written and there will be corruption. You should always stop defragmenting before removing a drive.
Also, if you dont have a removable drive set for quick removal in Windows, pulling it out before unmounting it can cause file damage as well.