Your guess is right. Windows XP has CPU throttling built in to the OS and does not provide easy access to changing it. However, you can use the built-in command-line POWERCFG.EXE utility to change it - see a discussion at Technet here or a more complete explanation on Wikipedia here. There's a tutorial for it here.
In short, to get maximum CPU speed while on battery for a Power Scheme named "Portable/Laptop" use the command:
C:\Windows\System32\PowerCFG.exe /change "Portable/Laptop" /processor-throttle-dc none
To switch it back change "none" to "dynamic" (or whatever its previous setting was). You can check the power settings by using this command:
C:\Windows\System32\Powercfg /query
You can also see the actual speed you're getting in each situation by looking at My Computer --> Properties. Under the Computer section you'll see the name of your CPU and it's maximum frequency. Under that it will show the actual frequency it's running at. Check under both conditions (battery & plugged in) and you should see the difference. Then change it with POWERCFG and see the difference both there & in your benchmarks.
FYI, there is an alternate tool you can try instead of POWERCFG called SpeedswitchXP - I have no experience with it, but it's described here. According to the page it gives GUI access to the CPU throttling option.