If the program won't ever need to be changed from within the PE environment, you might as well put it inside the Windows image (the mount folder if you're following the TechNet tutorial; all that stuff gets jammed into media\sources\boot.wim). That way, it will appear to be on the system drive and, like all the other stuff there, not persist across reboots. It doesn't really matter where you put the executable in there. I've seen people create subfolders in the Windows directory to hold third-party apps.
Usually, the Windows environment the flash drive boots to assigns a letter to the actual flash drive. If you need to modify the executable in the PE environment or put things next to it, you'll want to put the program on the drive after you get the PE stuff onto it.