InnoDB insert count slow down

Question

I batch insert to mysql innoDB table continuously, insert per second ratio is slow down. Some behaviour are - If shutdown data inserter (java) application, mysql do some i/o operation for a while. - Add some insert then shutdown mysql server, shutdown operation duration is too long. If start and stop mysql without any insertion, start and stop operation so fast. - Insert speed is not (so much) depend on data amount on table. If restart mysql server, insert per second is similar to last restart insert per second value.

I read some comment on forum, do not add continuosly, have gap between 2 insertions. Is it meaningful? Why sql is slow down?

query SHOW VARIABLES LIKE 'inno%' result is below

innodb_adaptive_flushing = ON
innodb_adaptive_hash_index = ON
innodb_additional_mem_pool_size = 20971520
innodb_autoextend_increment = 8
innodb_autoinc_lock_mode = 1
innodb_buffer_pool_instances = 1
innodb_buffer_pool_size = 268435456
innodb_change_buffering = all
innodb_checksums = ON
innodb_commit_concurrency = 0
innodb_concurrency_tickets = 500
innodb_data_file_path = ibdata1:50M:autoextend
innodb_data_home_dir = 
innodb_doublewrite = ON
innodb_fast_shutdown = 1
innodb_file_format = Barracuda
innodb_file_format_check = ON
innodb_file_format_max = Antelope
innodb_file_per_table = ON
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DSYNC
innodb_force_recovery = 0
innodb_io_capacity = 200
innodb_lock_wait_timeout = 50
innodb_locks_unsafe_for_binlog = OFF
innodb_log_buffer_size = 8388608
innodb_log_file_size = 268435456
innodb_log_files_in_group = 2
innodb_log_group_home_dir = ./
innodb_max_dirty_pages_pct = 75
innodb_max_purge_lag = 0
innodb_mirrored_log_groups = 1
innodb_old_blocks_pct = 37
innodb_old_blocks_time = 0
innodb_open_files = 300
innodb_purge_batch_size = 20
innodb_purge_threads = 0
innodb_read_ahead_threshold = 56
innodb_read_io_threads = 4
innodb_replication_delay = 0
innodb_rollback_on_timeout = OFF
innodb_spin_wait_delay = 6
innodb_stats_on_metadata = ON
innodb_stats_sample_pages = 8
innodb_strict_mode = ON
innodb_support_xa = ON
innodb_sync_spin_loops = 30
innodb_table_locks = ON
innodb_thread_concurrency = 0
innodb_thread_sleep_delay = 10000
innodb_use_native_aio = OFF
innodb_use_sys_malloc = ON
innodb_version = 1.1.1
innodb_write_io_threads = 4

Thanks

score 0 · Answer 1 · answered Jul 15 '11 at 12:58

0

InnoDB works by default in autocommit mode, which means every insert requires writing to disk twice. Using extended inserts (a.k.a. multi-row inserts) and enclosing several consecutive inserts into a transaction increases performance.

answered Jul 15 '11 at 12:58

Mchl

61,444
9
118
120

But are you commiting after eery single commit, or after a larger batch? – Mchl Jul 15 '11 at 13:59
1

BTW: The fastest way to bulk upload large amount of data is to use `LOAD DATA INFILE` – Mchl Jul 15 '11 at 14:08
code is add multiple insert. Actually problem is not about speed, it's about slow down of speed. – Erdinç Taşkın Jul 15 '11 at 14:23
Would you mind running this query `SHOW VARIABLES LIKE 'inno%';` and updating your question with it's results? – Mchl Jul 15 '11 at 14:32

score 0 · Accepted Answer · answered Jul 21 '11 at 06:55

Slow down reason is insert operation stored on to cache (dirty page) and periodically written hard-drive. Until dirty page memory is full insert operations are fast then cache is full and insert speed bottleneck disc written (I/O). You can use below sql to show dirty page size "Modified db pages"

show engine innodb status

InnoDB insert count slow down

2 Answers2