I posted a related question on askubuntu.com, but this may be a better place and way to ask it.
I'm working on a way to speed up a db import. Currently the import is taking a week, give or take a few days, which is unacceptable, and I know it can happen faster (dumping the data from a production server takes under an hour).
I set up a MySQL database as a symlink on a ramdisk I set up using
ramfs, which is like a cacheless version of
tmpfs. Despite the performance difference between the a hard disk and the RAM, I'm still only seeing between 40-190 QPS.
The table is innodb.
How can I speed up the import?
I seem to be getting a lot of down votes on this question, so perhaps it would be helpful to clarify my intentions:
- This is not a production server
- I have a system of flushing the ramdisk to disk, and went through the effort to set it up so that I could do it at will
- My goal is to be able to import over 8 million records in a reasonable amount of time. Something less than a week would be nice, which is the performance I was getting with the default install of MySQL.