1

I am trying to debug a FTP server. I am currently getting

$ sudo /usr/sbin/vsftpd
500 OOPS: SSL: cannot load RSA private key

FTP connections are being refused. This is that status of from systemctl:

$ sudo systemctl status vsftpd.service
● vsftpd.service - vsftpd FTP server
   Loaded: loaded (/lib/systemd/system/vsftpd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2019-05-15 20:40:34 UTC; 7min ago
  Process: 3220 ExecStart=/usr/sbin/vsftpd /etc/vsftpd.conf (code=exited, status=2)
  Process: 3217 ExecStartPre=/bin/mkdir -p /var/run/vsftpd/empty (code=exited, status=0/SUCCESS)
 Main PID: 3220 (code=exited, status=2)

May 15 20:40:34 ip-10-0-0-27 systemd[1]: Stopped vsftpd FTP server.
May 15 20:40:34 ip-10-0-0-27 systemd[1]: Starting vsftpd FTP server...
May 15 20:40:34 ip-10-0-0-27 systemd[1]: Started vsftpd FTP server.
May 15 20:40:34 ip-10-0-0-27 systemd[1]: vsftpd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 15 20:40:34 ip-10-0-0-27 systemd[1]: vsftpd.service: Unit entered failed state.
May 15 20:40:34 ip-10-0-0-27 systemd[1]: vsftpd.service: Failed with result 'exit-code'.

Somethings is likely wring with the key file, but I don't know why. It looks normal to me:

$ sudo ls -l /etc/ssl/private/wildcard.key
-r-------- 1 root root 1679 May 15 20:38 /etc/ssl/private/wildcard.key

And contains

-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----

Contents of /etc/vsftpd.conf:

use_localtime=YES
hide_ids=YES

# Logging
dual_log_enable=YES
xferlog_enable=YES
log_ftp_protocol=YES
debug_ssl=YES

#listen_ipv6=YES
listen=YES

# Local users
anonymous_enable=NO
write_enable=YES
local_enable=YES
chroot_local_user=YES
allow_writeable_chroot=YES
secure_chroot_dir=/run/vsftpd/empty
user_sub_token=$USER
local_root=/home/$USER/incoming

# TLS/SSL

ssl_enable=YES
force_local_data_ssl=YES
force_local_logins_ssl=YES
rsa_cert_file=/etc/ssl/private/wildcard.crt
rsa_private_key_file=/etc/ssl/private/wildcard.key
ssl_sslv2=NO
ssl_sslv3=YES
ssl_tlsv1=YES

ssl_ciphers=ECDHE-RSA-AES256-GCM-SHA384:AES256-SHA
strict_ssl_read_eof=NO
ssl_request_cert=NO
require_ssl_reuse=NO

# Passive mode
pasv_enable=YES
pasv_address=[REDACTED]
pasv_min_port=50000
pasv_max_port=50099

Output of strace:

$strace /usr/sbin/vsftpd /etc/vsftpd.conf
execve("/usr/sbin/vsftpd", ["/usr/sbin/vsftpd", "/etc/vsftpd.conf"], [/* 22 vars */]) = 0
brk(0)                                  = 0x7f3c864aa000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=24984, ...}) = 0
mmap(NULL, 24984, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3c85d15000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libwrap.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p-\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=36632, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d14000
mmap(NULL, 2134176, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c856c7000
mprotect(0x7f3c856cf000, 2093056, PROT_NONE) = 0
mmap(0x7f3c858ce000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7f3c858ce000
mmap(0x7f3c858d0000, 160, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3c858d0000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpam.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=55856, ...}) = 0
mmap(NULL, 2150904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c854b9000
mprotect(0x7f3c854c6000, 2093056, PROT_NONE) = 0
mmap(0x7f3c856c5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x7f3c856c5000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libssl.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240.\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=387272, ...}) = 0
mmap(NULL, 2482576, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c8525a000
mprotect(0x7f3c852af000, 2097152, PROT_NONE) = 0
mmap(0x7f3c854af000, 40960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x55000) = 0x7f3c854af000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\356\5\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1938752, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d13000
mmap(NULL, 4049080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c84e7d000
mprotect(0x7f3c85030000, 2097152, PROT_NONE) = 0
mmap(0x7f3c85230000, 155648, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b3000) = 0x7f3c85230000
mmap(0x7f3c85256000, 14520, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3c85256000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \26\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=18952, ...}) = 0
mmap(NULL, 2114160, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c84c78000
mprotect(0x7f3c84c7c000, 2093056, PROT_NONE) = 0
mmap(0x7f3c84e7b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f3c84e7b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P \2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1857312, ...}) = 0
mmap(NULL, 3965632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c848af000
mprotect(0x7f3c84a6d000, 2097152, PROT_NONE) = 0
mmap(0x7f3c84c6d000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1be000) = 0x7f3c84c6d000
mmap(0x7f3c84c73000, 17088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3c84c73000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnsl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`A\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=97296, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d12000
mmap(NULL, 2202328, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c84695000
mprotect(0x7f3c846ac000, 2093056, PROT_NONE) = 0
mmap(0x7f3c848ab000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7f3c848ab000
mmap(0x7f3c848ad000, 6872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3c848ad000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libaudit.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240(\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=104936, ...}) = 0
mmap(NULL, 2241056, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c84471000
mprotect(0x7f3c8448a000, 2093056, PROT_NONE) = 0
mmap(0x7f3c84689000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18000) = 0x7f3c84689000
mmap(0x7f3c8468b000, 37408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3c8468b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14664, ...}) = 0
mmap(NULL, 2109744, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3c8426d000
mprotect(0x7f3c84270000, 2093056, PROT_NONE) = 0
mmap(0x7f3c8446f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f3c8446f000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d11000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d10000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d0e000
arch_prctl(ARCH_SET_FS, 0x7f3c85d0e740) = 0
mprotect(0x7f3c84c6d000, 16384, PROT_READ) = 0
mprotect(0x7f3c8446f000, 4096, PROT_READ) = 0
mprotect(0x7f3c84689000, 4096, PROT_READ) = 0
mprotect(0x7f3c848ab000, 4096, PROT_READ) = 0
mprotect(0x7f3c84e7b000, 4096, PROT_READ) = 0
mprotect(0x7f3c85230000, 110592, PROT_READ) = 0
mprotect(0x7f3c854af000, 12288, PROT_READ) = 0
mprotect(0x7f3c856c5000, 4096, PROT_READ) = 0
mprotect(0x7f3c858ce000, 4096, PROT_READ) = 0
mprotect(0x7f3c85d1c000, 4096, PROT_READ) = 0
mprotect(0x7f3c85af3000, 4096, PROT_READ) = 0
munmap(0x7f3c85d15000, 24984)           = 0
brk(0)                                  = 0x7f3c864aa000
brk(0x7f3c864cb000)                     = 0x7f3c864cb000
open("/etc/vsftpd.conf", O_RDONLY|O_NONBLOCK) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1177, ...}) = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d19000
mprotect(0x7f3c85d1b000, 4096, PROT_NONE) = 0
mprotect(0x7f3c85d19000, 4096, PROT_NONE) = 0
read(3, "# General. See http://vsftpd.bea"..., 1177) = 1177
mprotect(0x7f3c85d19000, 4096, PROT_READ) = 0
munmap(0x7f3c85d19000, 12288)           = 0
close(3)                                = 0
stat("/etc/vsftpd.conf", {st_mode=S_IFREG|0644, st_size=1177, ...}) = 0
getuid()                                = 0
getuid()                                = 0
getpid()                                = 7409
open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 3
fstat(3, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
poll([{fd=3, events=POLLIN}], 1, 10)    = 1 ([{fd=3, revents=POLLIN}])
read(3, "\204\30>\303\fE\234\240VU\233\10\313\361\354^\217@\231\367`\274\260\241\357\234u\211aR^T", 32) = 32
close(3)                                = 0
getuid()                                = 0
open("/etc/ssl/private/wildcard.crt", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0400, st_size=8242, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d1b000
read(3, "-----BEGIN CERTIFICATE-----\nXXXX"..., 4096) = 4096
read(3, "XXX..."..., 4096) = 4096
read(3, "XXX...=\n-----EN"..., 4096) = 50
read(3, "", 4096)                       = 0
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x7f3c85d1b000, 4096)            = 0
open("/etc/ssl/private/wildcard.key", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0400, st_size=1704, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3c85d1b000
read(3, "-----BEGIN PRIVATE KEY-----\nXXXX"..., 4096) = 1704
close(3)                                = 0
munmap(0x7f3c85d1b000, 4096)            = 0
fcntl(0, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
fcntl(0, F_SETFL, O_RDWR|O_NONBLOCK|O_LARGEFILE) = 0
write(0, "500 OOPS: ", 10500 OOPS: )              = 10
write(0, "SSL: cannot load RSA private key", 32SSL: cannot load RSA private key) = 32
write(0, "\r\n", 2
)                     = 2
exit_group(2)                           = ?
+++ exited with 2 +++
  • 1
    Do you have vsftpd specific logs ? The service may be running on another user and don't have the right to read the file – Dom May 16 at 11:27
  • Please provide /etc/vsftpd.conf in your question – Sergey Nudnov May 17 at 5:48
  • @SergeyNudnov Updated the post! – Harald Nordgren May 17 at 8:30
2

I have made some checks.

Your private key is in a wrong format.

vsftpd expects it to be a PEM encoded PKCS#8 format private key.

You have it as PEM encoded PKCS#1 format private key. To convert it into the correct format, use the following commands:

cd /etc/ssl/private/
openssl pkcs8 -topk8 -nocrypt -in wildcard.key -out wildcard_new.key
mv -f wildcard_new.key wildcard.key

Now your key should be like following:

-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----

Check your key and certificate match

Run the following commands to check that your private key and certificate match:

cd /etc/ssl/private/
openssl pkey -in wildcard.key -pubout -outform pem | sha256sum
openssl x509 -in wildcard.crt -pubkey -noout -outform pem | sha256sum

The commands should output the same hash values

  • Thanks for you comment, but unfortunately it didn't help in this case. Same problem still! – Harald Nordgren May 20 at 12:33
  • @HaraldNordgren Could you please run sudo strace /usr/sbin/vsftpd and add its output in your question – Sergey Nudnov May 20 at 14:49
  • Updated the question with the strace info! – Harald Nordgren May 21 at 8:36
  • @HaraldNordgren I have added into my answer commands to check that the key and certificate match. Also your certificate file is quite large. Is that a bundle with intermediate certificates? If so, your server certificate should be the first, and intermediate certificates should follow. – Sergey Nudnov May 21 at 12:15
  • @HaraldNordgren If you wildcard.crtcontains the private key too, it should be the first, and in that case you don't need to specify rsa_private_key_file in vsftpd.conf, or it should point to /etc/ssl/private/wildcard.crt too – Sergey Nudnov May 21 at 12:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.