How to speed up a data loading into InnoDB (LOAD DATA INFILE)?
I can recommend these settings to improve load time:
- innodb_doublewrite = 0
- innodb_support_xa = 0
- innodb_buffer_pool_size = (50-80% of system memory)
- innodb_log_file_size = (a large number - 256M etc)
- innodb_flush_log_at_trx_commit = 0
Other than settings, there are some things you can do yourself:
- Create indexes after loading (this is a new optimization with 5.5 / InnoDB plugin).
- Sort the data file before loading.
- Split the data file, and load in parallel.
Try removing indexes and triggers. You can re-create them after the load. Also look into using some of the high-load settings in my-huge.cnf instead of the defaults.
Some more innodb performance settings:
http://www.mysqlperformanceblog.com/2007/11/01/innodb-performance-optimization-basics/
This might not be exactly what you're looking for but is a trick I've used in the past
ALTER TABLE TABLE_NAME DISABLE KEYS;
LOAD DATA INFILE ... ;
ALTER TABLE TABLE_NAME ENABLE KEYS;
Hope it helps.