Is it faster to create indexes before or after data loading in MySQL?

Question

I have a data replication process that drops and recreates a few tables in a target database, then loads them up with data from a source database (running on another host, but that is immaterial to the question at hand).

The target database does need primary keys and a few other indexes on its tables, but not during the data loading. I'm currently loading all of the data, then creating the indexes. However, index creation takes a pretty long time--30 minutes of my data loader's 5 and a half hour running time.

My intuition tells me that creating the indexes at the end should be faster than creating them first, since the index would need to be rewritten with each insert.

Can anyone tell me for sure which way is faster? FWIW, I'm running MySQL 5.1 with InnoDB tables.

Answer 1

Your intuition is correct AFAIK. Is the table ordered? http://kevin.vanzonneveld.net/techblog/article/improve_mysql_insert_performance/ Had some suggestions for optimizing bulk inserts.

Answer 2

Here is something else to think about.

For all MyISAM tables, it is best to bulk load by disabling non-Unique indexes

ALTER TABLE myisamtabletoload DISABLE KEYS;
INSERT INTO myisamtabletoload ... ;
ALTER TABLE myisamtabletoload ENABLE KEYS;

Primary Keys and Unique Keys All Populated and Ordered During the INSERT.

Once ENABLE KEYS is executed, all non-Unique indexes are built via Sorting.

Unfortunately, DISABLE KEYS and ENABLE KEYS does not work for InnoDB because of the clustered key structures under its hood.