1

I have a self hosted Elasticsearch 6.2 cluster (2 master nodes, ~200Gb data each).

I plan to move to AWS Elasticsearch service & it's not possible to ssh into it.

What's the fastest way to move all indices from an old ES cluster to the cloud one?

On a self hosted ES I could copy the indices folder to a new ES & that's it.

  • You can use logstash to read from one and write to the other – TheFiddlerWins May 29 '18 at 18:42
2

Use a tool for dumping & restoring Elasticsearch data, like Elasticdump (https://www.npmjs.com/package/elasticdump).

You can easily use bash to transfer all of the indexes from one instance to another, like so:

old_instance="http://old_address:9200"
new_instance="http://new_address:9200"

es_indexes=$(curl -s "${old_instance}/_cat/indices" | awk '{ print $3 }')

for index in $es_indexes; do
  elasticdump \
    --input="${old_instance}/${index}" \
    --output="${new_instance}/${index}" \
    --type=mapping

  elasticdump \
    --input="${old_instance}/${index}" \
    --output="${new_instance}/${index}" \
    --type=data
done
  • Will it work for AWS ES though? It requires v4 signing of requests which is one of my concerns – GTXBxaKgCANmT9D9 May 29 '18 at 18:51
  • I unfortunately don't know - I don't use their managed ES. The tool has command line flags for specifying AWS credentials so I guess it's worth a try ;) – Wiesław Herr May 29 '18 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.