Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

[Process:]

Importing delimited text files from a Windows based server to a MySQL 5.1.41 instance (multiple databases) on a single Ubuntu 10.04.2 host.

The process is initiated on the Windows server via the MySQL exe using ‘load data local infile’.

There are 20 databases total and we import 15 files per database – 1 file per table.

All tables use the MyISAM engine.

Prior to each import, we truncate the destination tables.

All processing is done sequentially.

[Issues:]

The process will run fine for about 2 weeks then continues to increase significantly in overall processing time. For example, we’ll see a 35 minute run-time for 2 weeks, then the next day its 55 minutes, then 80 minutes, etc, then without any changes, it will drop back to 35 after about a week (length of times vary.) It does not seem to be the result of any other processing requirements as the server is basically dedicated to MySQL only and there are no manually added CRON tasks.

Even the truncate table steps appear to be affected.

An comparison of ‘show status’ on a normal day vs a slow day does not appear to show any major issues other than a high volume of aborted_connects but this variable seems to increase regularly, outside of the actual import processing window.

Not sure if this is some sort of MySQL resource buildup or something related to the OS.

Any input would be greatly appreciated.

share|improve this question
add comment

1 Answer 1

Not sure if this will help or not but have you investigated Multiple Key Caches?

share|improve this answer
    
thanks for the tip - I'll check that out. –  af-at-work Aug 5 '11 at 16:22
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.