Encrypting the drive itself would probably be okay -- you'd just have to unlock the drive/partition before Time Machine would be able to access it.
Honestly, though, you asked Mac OS to use an encrypted backup, and encryption takes extra time.... so... just let it do its thing and let it keep your data safe. If you have files that are too large and modified too often to be backed-up efficiently using Time Machine to the encrypted disk, consider making special arrangements for those files (and exclude them from TM) so they don't bog-down your TM backup process.