mysql5.7,启用基于logical_clock的多线程复制,发现error日志增长很快,查看日志发现大量关于多线程复制的Note级别日志。
2018-07-03T03:22:01.638371+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 298; events assigned = 3043329; worker queues filled over ove rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725810947700 waited (count) when Workers occupied = 5346 wait ed when Workers occupied = 0 2018-07-03T03:24:22.589147+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 141; events assigned = 3044353; worker queues filled over ove rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725810947700 waited (count) when Workers occupied = 5346 wait ed when Workers occupied = 0 2018-07-03T03:27:00.554437+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 158; events assigned = 3045377; worker queues filled over ove rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725818557700 waited (count) when Workers occupied = 5346 wait ed when Workers occupied = 0 2018-07-03T03:32:00.079699+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 300; events assigned = 3047425; worker queues filled over ove rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725846344900 waited (count) when Workers occupied = 5346 wait ed when Workers occupied = 0 2018-07-03T03:34:04.567887+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 124; events assigned = 3048449; worker queues filled over ove rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725852036800 waited (count) when Workers occupied = 5346 wait ed when Workers occupied = 0
通过日期可以看到,日志的打印频率大概在3分钟左右。error log是mysql 出现问题时的重要分析日志,大量的note日志可能很快就把重要日志信息埋没,给分析日志信息带来了不便。而且日志量随时间递增,还必须有rotate机制,不然占用大量磁盘空间。那么这个日志的打印频率能不能调低一些呢?
我看了下源码这块的实现(rpl_slave.cc 4800)
*ptr_ev= NULL; // announcing the event is passed to w-worker if (rli->is_parallel_exec() && rli->mts_events_assigned % 1024 == 1) { time_t my_now= my_time(0); if ((my_now - rli->mts_last_online_stat) >= mts_online_stat_period) { sql_print_information("Multi-threaded slave statistics%s: " "seconds elapsed = %lu; " "events assigned = %llu; " "worker queues filled over overrun level = %lu; " "waited due a Worker queue full = %lu; " "waited due the total size = %lu; " "waited at clock conflicts = %llu " "waited (count) when Workers occupied = %lu " "waited when Workers occupied = %llu", rli->get_for_channel_str(), static_cast(my_now - rli->mts_last_online_stat), rli->mts_events_assigned, rli->mts_wq_overrun_cnt, rli->mts_wq_overfill_cnt, rli->wq_size_waits_cnt, rli->mts_total_wait_overlap, rli->mts_wq_no_underrun_cnt, rli->mts_total_wait_worker_avail); rli->mts_last_online_stat= my_now;
我们从两个if语句可以看到,这个信息的打印受三个条件的控制:
- rli->is_parallel_exec() 这是个inline函数,它判断是否在使用多线程复制
- rli->mts_events_assigned % 1024 == 1 多线程执行的event个数刚刚超过1024个。也就是说按evnet个数来算,每隔1024个event打印一次。
- 最后还有个时间限制:(my_now - rli->mts_last_online_stat) >= mts_online_stat_period,距离上一次统计超过既定的间隔mts_online_stat_period。这个变量是60*2 ,代码里写死了。
/* Statistics go to the error log every # of seconds when --log-warnings > 1 */ const long mts_online_stat_period= 60 * 2;
由此可以判断,这个日志打印频率最少为2分钟,没法改了。但上面的代码注释中给了提示,当--log-warnings > 1时,这个统计间隔才生效,意味着可以通过修改日志打印级别来控制。
自mysql5.7.2,log-warnings被废弃,引用了新的变量log_error_verbosity。这个变量有三个值:1、2、3,默认是3. 他们的意义是:
1 -- Errors Only 2 -- Errors and warnings 3 -- Errors, warnings, and notes我们可以修改这个变量为2 ,不再打印Note信息。
set global log_error_verbosity=2;
可以写到配置文件中去
参考文档:
原文地址: