0

I am trying to create a AWS DMS (Data Migration Service) task to migrate the data from a RDS Read Replica running Postgres to a Redshift cluster, however my task is always failing when it is time to migrate a few tables (usually the ones that are more frequently in use fail).

I created a snapshot from this Read Replica and I was able to successfully migrate it to the Redshift cluster, so I don't think that the problem is related to the database itself.

Please note that I am using the read replica as my source, because my master database is located in a different region from the Redshift Cluster.

What I am doing wrong?

Improve this question
1
  • Does DMS show any error messages when it fails? That may give some insight into what's wrong.
    – MLu
    Jan 3, 2019 at 22:01

1 Answer 1

Reset to default
1

I found out what the problem was: since I am using the DMS with a read replica, if I have a long running query to transfer the data to the Redshift, this query will get cancelled when the read replica is updated.

So I changed the values of the: max_standby_archive_delay and max_standby_streaming_ delay to 15min (900000ms), this way Postgres will wait up to 15min before applying an update to the read replica, which is enough time for me to transfer the data from the Read Replica to the Redshift.

Please note that it is not possible to create an onging data transfer from a Read Replica. It is either a full load or nothing.

Reference doc: https://www.postgresql.org/docs/9.0/hot-standby.html

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .