There is one systemic method of protecting yourself that comes to mind, which I have not seen mentioned.
Host your Linux instances as VM's on a virtualization hypervisor (VMware, Xenserver, Hyper-V, etc.).
DO NOT give the remote admin administrative access to the hypervisor. The remote admin would only get root access to the VM's themselves.
DO Implement a hypervisor based backup system (Unitrends, Veeam, vSphere Data Protection, etc.)
DO keep at least one snapshot per day of each Linux VM, going back as far in time as you feel is necessary.
DO NOT give the remote admin write access to the backup repositories.
If you do these things, you will have backup snapshots of each Linux instance over which the remote admin has no control. If the remote admin does something hinky, whether intentionally or accidentally, you can always mount a backup from before the hinkeness occurred to evaluate what happened and possibly recover to a clean state.
This won't be proof against a hypervisor side-channel attack, which could potentially be mounted from within a VM that the attacker has root access to.
If your backups don't go far enough back in time, this won't protect you.
You need to thoroughly trust whomever is in control of your hypervisor and the backup infrastructure.
If you're doing this in the cloud (AWS, Azure, etc.), the implementation details will differ, but the general concept would be the same.
In essence, divide responsibilities among parties who are not business partners with each other, in addition to only hiring people you trust.