If I recall correctly, "non-system disk" is a common error message that's displayed when trying to boot, in BIOS mode, some types of disks that lack BIOS-mode boot loaders. If my memory and analysis are correct, then, rEFIt has tried to boot your flash drive in BIOS mode, but it's not bootable in that way.
Without knowing what version of Ubuntu you're trying to install (both the version number and the type of image you obtained, as in desktop, alternate, Mac-specific, etc.) and how you prepared the flash drive (dd, file-level copy, etc.), it's hard to offer specific recovery recommendations except to go back to the Ubuntu page and follow its instructions precisely on how to prepare your boot medium.
One more comment: Most sources that describe how to install Ubuntu on a Mac describe setting up Ubuntu to boot in BIOS mode. This works, but it's got some serious drawbacks, particularly if Windows is not installed. Specifically, you're forced to use a hybrid MBR, which is a dangerous and standards-violating hack. If possible, it's much better to avoid using a hybrid MBR, but on a Mac, this requires booting in EFI mode rather than in MBR mode. Such a configuration is possible, but it requires setting things up a bit differently, and EFI-mode installs fail to activate some hardware on some models, so it doesn't work acceptably for everybody. I've written a Web page on the topic if you're interested in giving it a try.