Importing delimited text files from a Windows based server to a MySQL 5.1.41 instance (multiple databases) on a single Ubuntu 10.04.2 host.
The process is initiated on the Windows server via the MySQL exe using ‘load data local infile’.
There are 20 databases total and we import 15 files per database – 1 file per table.
All tables use the MyISAM engine.
Prior to each import, we truncate the destination tables.
All processing is done sequentially.
The process will run fine for about 2 weeks then continues to increase significantly in overall processing time. For example, we’ll see a 35 minute run-time for 2 weeks, then the next day its 55 minutes, then 80 minutes, etc, then without any changes, it will drop back to 35 after about a week (length of times vary.) It does not seem to be the result of any other processing requirements as the server is basically dedicated to MySQL only and there are no manually added CRON tasks.
Even the truncate table steps appear to be affected.
An comparison of ‘show status’ on a normal day vs a slow day does not appear to show any major issues other than a high volume of aborted_connects but this variable seems to increase regularly, outside of the actual import processing window.
Not sure if this is some sort of MySQL resource buildup or something related to the OS.
Any input would be greatly appreciated.