I am working with a group of servers on a VPN with ubuntu 18.04, and for each server I have a different key file. However, when I try to securely copy files (like securely propagate munge key across servers) I am denied as I don't have an appropriate key (at least I think that is the reason, it returns "Permission denied (publickey).").
Keeping my private keys on each server is probably a bad practice? Copying files to my local machine and then copying them to a different server is a pain, and I am not sure if that is secure either.
How is this properly handled? Can I use a user on each server (like munge) to copy between them, if not to any location then to some designated folders which I can later sudo cp? Is there an option that is not a vulnerability?