It is not different than hiring a sys admin at some point. Good hosters hae procedures in place and documented and contractually follow them.
Also, this is wrong:
it is generally admitted that a computer can be considered unsecure when
someone else than the designated operator has physical access to it.
If with generally you mean "outdated technology" - yes. But TPM and a lot of modern technology is there to particularly make it hard for even an admin to do bad things.
Example of such things are for example "Hyper-V Shielded Virtual Machine". EPYC processors for example are capable of keeping the memory encrypted (iirc on a per virtual machine basis), so that even as admin you can not really read a VM's memory with a debugger.
Now, you likely always have AN attack vector, but proper technologies mean that not everyone with access to the physical computer can actually read the data from virtual machines - you will need a MUCH higher access level that can be limited to very few people.