Insert-heavy InnoDB table won't use all my CPU

You have to crank up innodb_io_capacity as well.

The default is 200. Raise it to 5000 for starters. I would go to 20000.

You may also want to make sure ib_logfile0 and ib_logfile1 are sufficiently large. The default value for innodb_log_file_size is 5M. I would raise that to 1G for starters.

A larger InnoDB Buffer Pool would also help, perhaps 4G.

To recap, use these additional settings:

[mysqld]
innodb_io_capacity=5000
innodb_buffer_pool_size=4G
innodb_log_file_size=1G

After adding these settings to my.cnf, to resize ib_logfile0/ib_logfile1 do the following

service mysql stop
rm -f /var/log/mysql/ib_logfile[01]
service mysql start

The files ib_logfile0 and ib_logfile1 are recreated. Don't worry, I have done this many times.

You may have to do something out of the ordinary for InnoDB

Try the following:

  • Full Table Lock on the InnoDB table
  • Perform the Bulk Load
  • Release the Lock

There are a number of factors that impact the ability to maximize use of multiple cores.

  • Some mutexes will impact multiple CPUs leaving some waiting before they can proceed.
  • You need as many active threads as you have CPUs. If your workload results in 9 parallel thread, you can't fill 12 cores.
  • I/O capacity must be sufficient to provide enough work for all CPUs. If you are queueing on disk I/O or waiting for network messages, then you won't be able to fill the CPUs.

Tools like SAR will enable you to determine if there are any bottlenecks which are reducing your capacity. Just be warned, eliminating one bottleneck, will just move the bottleneck.