Yes, this is to prevent brute force attacks.
An alphanumeric 6 character password can have up to 36 bits of entropy (6 bits per character). If a computer can check 1 billion passwords per second, it will need only 2 ^ 36 / 1 billion = 69 seconds to try all possible passwords. A delay of one second means that only one password can be tried per second. Trying all possible passwords would take 2179 years now...
The delay isn't handled by passwd itself, but by the Pluggable Authentication Modules.
The delay can be disabled for passwd by adding the option nodelay to the line
password [success=2 default=ignore] pam_unix.so obscure sha512 in /etc/pam.d/common-password. Since this would affect all other programs using that configuration file as well, you could copy it over /etc/pam.d/passwd and disable the delay only there.
See also: man pam_unix