You provide evidence that the USB drive is being used for ReadyBoost, but then wonder why it's not. Perhaps you are actually wondering why it's not resulting in a speed increase?
If so, this is probably because USB2 is capped at about 35 MB/sec, likely slower than your other drives. It could also be that your workload does not lend itself to easy caching, or that 3.5 GB doesn't make much difference. But fundamentally, you are unlikely to see a difference with a USB 2 drive used as a cache.