This is a security feature implemented by the PAM module that's checking your passwords.
If you read the manpage for pam_unix(8), you'll see a "nodelay" option. pam_unix is used by ssh to check passwords, by login, and anything else that checks your password. "The default action is for the module to request a delay-on-failure of the order of two second."
There's a function all the pam modules can call to request the main pam stuff delay. pam_fail_delay. This function takes the longest delay any module called and randomly changes it by up to 25% and waits that long before returning to the application (sshd, login, etc).
It's a common security measure on many systems that a failure (such as trying to check password for a user that doesn't exist) will insert a small sleep. This prevents timing attacks, since otherwise an unsuccessful login to a valid account could take slightly longer (encrypting password, stats in home directory, etc) than an attempt to log into a non-existent account. Other terms for this are covert channel, timing channel, or side channel attack. It also discourages brute force attacks (but not very well, just open more connections). Randomizing the sleep slightly helps, also.