0

I'm running some Django migrations on an AWS RDS MySQL instance. My team has tested the migrations very thoroughly and they ran fine (even taking less than 30 minutes when run on our laptops against a copy of our production data).

Now that we're migrating the production DB, it's taking over 2 hours. When I run:

mysql> SHOW PROCESSLIST;

I can see the active process and it is doing stuff, but very slowly. Our migrations are in the middle of a data migration, and the SELECT statements (we're fetching rows through the ORM in batches of 100) are taking up to 32 seconds each. The last 30 seconds are spent in a state of "removing tmp table".

Any idea what's causing this?

1 Answer 1

0

we're fetching rows through the ORM in batches of 100

Innefficient queries for batching the data. Since the DBMS is using temporary tables it is having to examine much, MUCH more data than is included in the output dataset - and most of it is being discarded.

Go over to StackOverflow and post your schema and the queries you are using to extract the data for more specific advice - but IME, ORM is so broken in principle that its usually easier to switch to factories than try to get to get it working efficiently/reliably.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .