Innodb: after 48 hours of optimizing 10mb/sec write speed
Suggestions to consider for your my.cnf [mysqld] section based on visible information provided. Entire block to go at END of [mysqld] and REMOVE any SAME NAMED VARIABLE appearing higher in the section to avoid conflicts on requests.
innodb_io_capacity=40000 # from 5000 to open the door for NVME speed
read_rnd_buffer_size=256K # from 1M to reduce handler_read_rnd_next RPS
innodb_lru_scan_depth=128 # from 1024 to conserve CPU every second
innodb_adaptive_max_sleep_delay=10000 # from 150000 for 1 sec sleep delay
innodb_flushing_avg_loops=4 # from 30 for reduce the loop delay
innodb_thread_concurrency=0 # from 144 see dba.stackexhange Question 5666
max_seeks_for_key=32 # to limit optimizer to nn vs ~ 4 Billion possible
max_write_locks_count=16 # to allow RD after nn lcks vs up to 4 Billion lcks
thread_concurrency=30 # from 10 for additional conc - may be DEPR
innodb_buffer_pool_instances=8 # from 64 see REFMAN for innodb_lru_scan_depth details
innodb_log_file_size=6G # from ~ 512M to reduce log rotation
innodb_log_buffer_size=3G # from 16M for ~ 30 minutes buffering
query_cache_type=OFF # from ON no need to waste CPU for mgmt
query_cache_size=0 # from ~256M to conserve RAM and CPU cycles
slow_query_log=ON # from OFF always good to have ON
RAM use strategy while you are loading 10,000+ rows per second
Total RAM = 64GB, allow mysqld up to 48GB (~ 75%)
While you are loading this high volume,
innodb_buffer_pool_size=30G # for 62.5% of your 48G
innodb_change_buffer_max_size=50 # for 50% to have best insert rate per second
when the loading has completed,
SET GLOBAL innodb_change_buffer_max_size=15 # for 15% set aside for routine maintenance requirements;
and you will settle into typical data needed available in the innodb buffer pool in a reasonable length of uptime.
For additional Suggestions, view my profile, Network Profile for contact info, including SKYPE ID to get in touch, PLEASE. Thanks