internal-sftp are part of OpenSSH.
sftp-server is a standalone binary.
internal-sftp is just a configuration keyword that tells
sshd to use the SFTP server code built-into
sshd, instead of running another process (what would typically be the
internal-sftp was added much later (OpenSSH 4.9p1 in 2008?) than the standalone
sftp-server binary. But it is the default by now.
sftp-server is now redundant and is kept for a backward compatibility.
I believe there's no reason to use the
sftp-server for new installations.
From functional point of view,
internal-sftp are almost identical. They are built from the same source code.
The main advantage of
internal-sftp is, that it requires no support files when used with
Quotes from the
sshd_config(5) man page:
Another advantage of
internal-sftp is a performance, as it's not necessary to run a new sub-process for it.
It may seem that
sshd could automatically use
internal-sftp, when it encounters
sftp-server, as the functionality is identical and
internal-sftp has even the above advantages. But there are edge cases, where there are differences.
Administrator may rely on a login shell configuration to prevent certain users from logging in. Switching to the
internal-sftp would bypass the restriction, as the login shell is no longer involved.
sftp-server binary (being a standalone process) you can use some hacks, like running the SFTP under
For SSH-1 (if anyone is still using it),
Subsystem directive is not involved at all. An SFTP client using SSH-1 tells the server explicitly, what binary the server should run. So legacy SSH-1 SFTP clients have
sftp-server name hard-coded.