I have searched for an answer, and found resources about changing the delay after an incorrect password entry for an SSH login, but not to my specific problem.
My problem is that the delay is after I enter my username, not password, and the delay is about 20 seconds. I don't mind a delay on incorrect password, but if I enter my username, I want to enter the password immediately after.
If it helps anyone, the same delay happens on FTP connect. However, once I'm connected (either SSH or FTP) then all interaction is lightning-fast. It's only the login that is slow.
I looked in /etc/ssh/sshd_config and do see this entry:
UsePam yes
But I don't know anything about PAM or where it's config is located.
[edit: here are the log files (last ones) from /var/log/auth.log]
Mar 17 14:27:29 rel2015 sshd[26206]: Failed password for root from 218.65.30.107 port 57695 ssh2
Mar 17 14:27:30 rel2015 sshd[26206]: Received disconnect from 218.65.30.107: 11: [preauth]
Mar 17 14:27:30 rel2015 sshd[26206]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.65.30.107 user=root
Mar 17 14:27:42 rel2015 sshd[26208]: Received disconnect from 218.65.30.107: 11: [preauth]
Mar 17 14:28:08 rel2015 proftpd: pam_unix(proftpd:session): session closed for user rbase
Mar 17 14:28:23 rel2015 proftpd: pam_unix(proftpd:session): session opened for user rbase by (uid=0)
Mar 17 14:29:25 rel2015 proftpd: pam_unix(proftpd:session): session closed for user cpm303
Mar 17 14:29:41 rel2015 proftpd: pam_unix(proftpd:session): session opened for user cpm303 by (uid=0)
Mar 17 14:35:01 rel2015 CRON[26318]: pam_unix(cron:session): session opened for user root by (uid=0)
Mar 17 14:35:01 rel2015 CRON[26318]: pam_unix(cron:session): session closed for user root
obv. using PAM, but again I don't know where PAM is located..
/var/log/auth.log
above