"Host key verification failed" on znapzendzetup create command

Question

Just getting started with znapzend which compiled fine on Ubuntu 14.04 (the source machine, since updated to Ubuntu 16.04). Destination machine is Ubuntu 16.04 ('backupserver'). I've setup a non-passworded ssh keypair with ssh-keygen/ssh-copy-id [destination]. I can ssh backupserver without password just fine. Problem is, znapzend cannot seem to reach backupserver for some reason. mbuffer is at /usr/bin/mbuffer on both machines. Here is my znapzendzetup command and output running on the source machine:

sudo znapzendzetup create --recursive --mbuffer=/usr/bin/mbuffer --tsformat='%Y-%m-%d-%H%M%S' SRC '1y=>1d' pool/Documents DST '1y=>1d' jon@backupserver:pool/Documents
Host key verification failed.
Host key verification failed.
*** WARNING: executable '/usr/bin/mbuffer' does not exist on jon@backupserver

*** backup plan: pool/Documents ***
dst_0           = jon@backupserver:pool/Documents
dst_0_plan      = 1year=>1day
*** WARNING: destination 'jon@backupserver:pool/Documents' does not exist, will be ignored! ***

enabled         = on
mbuffer         = /usr/bin/mbuffer
mbuffer_size    = 1G
post_znap_cmd   = off
pre_znap_cmd    = off
recursive       = on
src             = pool/Documents
src_plan        = 1year=>1day
tsformat        = %Y-%m-%d-%H%M%S

Do you want to save this backup set [y/N]?

Thanks for any insight you can provide!

---

UPDATE #1

I'm getting either:

Host key verification failed.

or

Permission denied the ZFS utilities must be run as root.

depending on if I run znapzendzetup with sudo, or I have --sudo in the command:

sudo znapzendzetup create ...

or

znapzendzetup create --sudo ...

I added this line to /etc/sudoers (via visudo):

jon ALL=(ALL) NOPASSWD:ALL

And executed this line on both the host and destination machines:

sudo zfs allow -u jon create,mount,send,receive,snapshot,destroy pool/Documents

I can try enabling password-less root login, but I understand that is discouraged. Thanks for taking a look!

---

UPDATE #2

I enabled remote root login over ssh by changing these lines in /etc/ssh/sshd_confg on the destination machine:

#PermitRootLogin prohibit-password
PermitRootLogin yes

Then, on the source machine:

sudo passwd root
su
ssh-keygen
ssh-copy-id BACKUPSERVER

znapzend commands now work as sudo, and without the --sudo argument!

Next, I initiated a znapzend command with the output below:

jon@FILESERVER:~$ sudo znapzend --debug --runonce=pool/Documents
[Wed Apr 27 17:54:42 2016] [info] znapzend (PID=6388) starting up ...
[Wed Apr 27 17:54:42 2016] [info] refreshing backup plans...
[Wed Apr 27 17:54:42 2016] [info] found a valid backup plan for pool/Documents...
[Wed Apr 27 17:54:42 2016] [info] znapzend (PID=6388) initialized -- resuming normal operations.
[Wed Apr 27 17:54:42 2016] [debug] snapshot worker for pool/Documents spawned (6391)
[Wed Apr 27 17:54:42 2016] [info] creating snapshot on pool/Documents
# zfs snapshot pool/Documents@2016-04-27-175442
[Wed Apr 27 17:54:42 2016] [debug] snapshot worker for pool/Documents done (6391)
[Wed Apr 27 17:54:42 2016] [debug] send/receive worker for pool/Documents spawned (6393)
[Wed Apr 27 17:54:42 2016] [info] starting work on backupSet pool/Documents
# zfs list -H -o name -t snapshot -s creation -d 1 pool/Documents
[Wed Apr 27 17:54:42 2016] [debug] cleaning up snapshots on pool/Documents
# zfs destroy pool/Documents@2016-04-27-174027
[Wed Apr 27 17:54:43 2016] [info] done with backupset pool/Documents in 1 seconds
[Wed Apr 27 17:54:43 2016] [debug] send/receive worker for pool/Documents done (6393)

It appears to have run successfully, but so far, it hasn't backed up the dataset to the destination yet. When does it copy files over?

Also, just FYI, I added znapzend as a service with:

sudo cp ~/znapzend-0.15.5/init/znapzend.service /etc/systemd/system
sudo chown root:root /etc/systemd/system/znapzend.service
systemctl enable znapzend.service
sudo systemctl enable znapzend.service
sudo systemctl start znapzend.service
sudo service znapzend status

Thanks!

---

UPDATE #3

Ok, I have this sorted now. I did not have the configuration setup to login with 'root' on the destination machine, which prevented send/receive to function. Here's my updated znapzendzetup command:

sudo znapzendzetup create --mbuffer=/usr/bin/mbuffer --tsformat='%Y-%m-%d-%H%M%S' SRC '7d=>1h,1y=>1d' pool/Documents DST '7d=>1h,1y=>1d' root@BACKUPSERVER:pool/Documents

The backup completed successfully after:

sudo znapzend --debug --runonce=pool/Documents

Thanks for this great ZFS snapshot+backup tool!!!

Answer 1

since you are using sudo I assume the keys you have setup are not valid for the right user ... try

sudo ssh backuphost ls

to verify your setup

Answer 2

Tobi got me pointed in the right direction (thanks!). My solution was to enable password-less root login over ssh, which probably isn't ideal, but I'm in a home LAN, so should be fine. The problem was, I couldn't get a non-root user to run zfs commands with sudo over ssh for some reason, even if password-less ssh login was fully working as that non-root user. 'znapzendzetup create --sudo ...' would work, but then 'znapzend --noaction --debug ' gave: 'Permission denied the ZFS utilities must be run as root' for some reason (and there isn't a --sudo option for the znapzend command I guess). So I setup ssh login as root (UPDATE #2). To check it, I did:

sudo ssh BACUPSERVER 'zfs list'

Then I had to remember to change my znapzendzetup command to login as root with (see UPDATE #3):

sudo znapzendzetup ... root@BACKUPSERVER:pool/Documents

Last night I added a '1w=>30min' snapshot interval to all three of my datasets, and this morning it was all backed up perfectly to my BACKUPSERVER. Nice!