Server returned 'request expired' less than 0 seconds after request was issued, but it shouldn't have expired for at least 600 seconds

Question

When I click save & restart VMWare Replication Appliance VRM service:

Getting this error Error applying startup configuration: Server returned 'request expired' less than 0 seconds after request was issued, but it shouldn't have expired for at least 600 seconds.

https://vrm.vmware.domain.local:5480/service/hms/cgi/health.py gives green in XML

LookupService Address: vcenter.vmware.domain.local (and it's pingable from ssh @ vrm.vmware.domain.loca, DNS is set in networkingl)
SSO Admin: [email protected] (Such SSO domain exists in vcenter and I can login in vcenter web UI with vmware.domain.local\Administrator)
VRM Host: vrm.vmware.domain.local (also pinged from multiple hosts)
VRM Site Name: VRM NPB
vcenter Server Address: vcenter.vmware.domain.local
vcenter server port: 80

If I click Start VRM Service, it gets into

VRM service is started and quickly reverts to
VRM service is stopped

Installation of vcenter6 and vsphere replication appliance is done from ISOs (appliances, latest downloaded from VMWare 2 days ago)

vcenter complains on HMS service: Failed to request health status from https://vrm.vmware.domain.local:5480/service/hms/cgi/health.py

(however in my browser I can easily open this URL, yet the certs are self signed)

What can be done?

I even debugged the script the VRM UI sends http GET https://vrm.vmware.domain.local:5480/service/hms/cgi/hms.py?cmd=checklscert&ls=vcenter.vmware.domain.local&lsthumbpr…

{
   "result":{
      "vc":{
         "uuid":"56d6e005-2229-4bc6-99e9-7969c8883efd",
         "notBefore":"26 Jun 2015 12:56:47 GMT",
         "notAfter":"20 Jun 2025 12:56:47 GMT",
         "valid":"Certificate validation error",
         "thumbprint":"8D:E6:70:13:D2:4A:2D:D6:D7:27:21:C3:FA:36:48:56:F3:EA:91:EA",
         "subject":{
            "C":"US",
            "CN":"vcenter.vmware.domain.local",
            "L":"Unknown",
            "O":"Unknown",
            "ST":"Unknown",
            "OU":"Unknown"
         },
         "issuer":{
            "C":"US",
            "CN":"CA",
            "DC":"vmware",
            "L":"Unknown",
            "O":"vcenter.vmware.domain.local",
            "ST":"Unknown",
            "OU":"Unknown"
         }
      },
      "result":"ok",
      "ls":{
         "notBefore":"26 Jun 2015 12:56:47 GMT",
         "notAfter":"20 Jun 2025 12:56:47 GMT",
         "valid":"true",
         "thumbprint":"8D:E6:70:13:D2:4A:2D:D6:D7:27:21:C3:FA:36:48:56:F3:EA:91:EA",
         "subject":{
            "C":"US",
            "CN":"vcenter.vmware.domain.local",
            "L":"Unknown",
            "O":"Unknown",
            "ST":"Unknown",
            "OU":"Unknown"
         },
         "issuer":{
            "C":"US",
            "CN":"CA",
            "DC":"vmware",
            "L":"Unknown",
            "O":"vcenter.vmware.domain.local",
            "ST":"Unknown",
            "OU":"Unknown"
         }
      }
   },
   "error":null
}

If I'm trying to unregister the appliance I get this error: The appliance has to be configured correctly for unregister to work.

Answer 1

the solution to fix this is to sync all parties (ESXi hosts, vcenter, VRM Appliance) with ntp servers. Once this done - the error is gone.