I currently, have one public elastic cluster with one data node(m3.xlarge.elasticsearch) which is working fine. I would like to move from the public domain to VPC based domain.
I was able to take a manual snapshot from the public endpoint and restore it to VPC endpoint.
While the migration is successful, I am currently facing issue with search latency while the public endpoint has the search latency of ~0.22-0.23ms the VPC endpoint with 2(m4.large.elasticsearch) data nodes has the search latency of ~0.600-1.5ms depending on the load.
This results in slowing down the entire website, while hooked up to public endpoint a typical page loads in about 6-7s while we move to VPC endpoint the page loads in about 20-30s which is as good as system down.
Another metric which I observed was in Newrelic. It groups external metrics and their response time. For the public endpoint, it was about ~8.92ms when I moved to VPC it was about ~180-230ms.
It seems to be a no-brainer, all anyone had to do was to migrate from public to VPC based cluster with the current generation nodes with better availability and should have resulted in better performance but the results are otherwise.
Apart from the suggested infra above have tried with the following architecture
- 3 T2.Medium(data-node)
- 3 Master nodes + 5 Data nodes.
Irrespective of the resources used, I am getting the same response time issue.
Any suggestions to answer such behavior would be great. Let me know if any more additional information required.