Skip to content

Commit

Permalink
mysql-innodb-buffer-pool-size: Improve code and output
Browse files Browse the repository at this point in the history
  • Loading branch information
@markuslf
markuslf committed Sep 22, 2023
1 parent 884a750 commit ea04942
Show file tree
Hide file tree
Showing 3 changed files with 69 additions and 36 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ Monitoring Plugins:
* infomaniak-swiss-backup-devices: Improve column ordering in output
* journald-query: Improve output
* mysql-aria: Remove WARN if `aria_pagecache_read_requests` > 0 and `pct_aria_keys_from_mem` < 95%
* mysql-innodb-buffer-pool-size: Improve code and output
* mysql-logfile: Stop magic auto-configure if `--server-log` is given
* mysql-logfile: Returns OK instead of UNKNOWN if logfile is found but empty
* mysql-logfile: State only UNKNOWN if the log is empty and wasn't set deliberately ([PR #716](https://github.com/Linuxfabrik/monitoring-plugins/issues/716), thanks to [Eric Esser](https://github.com/dorkmaneuver))
Expand Down
15 changes: 6 additions & 9 deletions check-plugins/mysql-innodb-buffer-pool-size/README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,7 @@ Overview

Checks the size of the InnoDB buffer pool in MySQL/MariaDB. Logic is taken from `MySQLTuner script <https://github.com/major/MySQLTuner-perl>`_:mysql_innodb().

Always take care of both ``innodb_buffer_pool_size`` and ``innodb_log_file_size`` when making adjustments. For that have a look at the following output example ``InnoDB buffer pool / data size: 36.0GiB / 49.4GiB [WARNING]. Set innodb_buffer_pool_size >= 49.4GiB. innodb_log_file_size * innodb_log_files_in_group / innodb_buffer_pool_size = 9.0GiB * 2 / 36.0GiB = 50.0% [WARNING] (should be 25%). Set innodb_log_file_size to 4.5GiB.``:

* Here, buffer pool is 36 GB.
* Data does not fit in, it needs 49 GB.
* The check plugin complains and makes some suggestions on how to resize ``innodb_buffer_pool_size`` and ``innodb_log_file_size``.
* If we adjust ``innodb_buffer_pool_size`` to 50 GB, and ``innodb_log_files_in_group`` is set to ``2`` (deprecated and ignored from MariaDB 10.5.2), we should set ``innodb_log_file_size`` to ``6.25`` to get the 25% log file versus pool size ratio. Then both warnings should change to OK.
* Attention: Assuming this is a database server with entirely/primarily InnoDB tables, you need at least 64 GB, following the rule that the InnoDB buffer pool size can be set up to 80% of the total memory in this case.
Always take care of both ``innodb_buffer_pool_size`` and ``innodb_log_file_size`` when making adjustments.

User account requires:

Expand Down Expand Up @@ -78,14 +72,17 @@ Output:

.. code-block:: text
InnoDB buffer pool / data size: 36.0GiB / 49.4GiB [WARNING]. Set innodb_buffer_pool_size >= 49.4GiB. innodb_log_file_size * innodb_log_files_in_group / innodb_buffer_pool_size = 9.0GiB * 2 / 36.0GiB = 50.0% [WARNING] (should be 25%). Set innodb_log_file_size to 4.5GiB.
Data size: 2.5GiB, innodb_buffer_pool_size: 4.0GiB
Ratio innodb_log_file_size (1.0GiB) * innodb_log_files_in_group (1) vs. innodb_buffer_pool_size (4.0GiB): 25%
States
------

* WARN on 32 bit systems when InnoDB buffer pool size > 4 GiB.
* WARN on 64 bit systems when InnoDB buffer pool size > 16 EiB.
* WARN if size of data does not fit into the InnoDB buffer pool.
* WARN if the InnoDB log file size versus the InnoDB pool size ratio is not in the range of 20 to 30%.
* WARN if the InnoDB log file size versus the InnoDB buffer pool size ratio is not in the range of 20 to 30%.


Perfdata / Metrics
Expand Down
89 changes: 62 additions & 27 deletions check-plugins/mysql-innodb-buffer-pool-size/mysql-innodb-buffer-pool-size
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ from lib.globals import (STATE_OK, STATE_UNKNOWN, # pylint: disable=C0413
STATE_WARN)

__author__ = 'Linuxfabrik GmbH, Zurich/Switzerland'
__version__ = '2023071203'
__version__ = '2023092201'

DESCRIPTION = """Checks the size of the InnoDB buffer pool in MySQL/MariaDB."""

Expand Down Expand Up @@ -80,21 +80,22 @@ def get_vars(conn):
# Do not implement `get_all_vars()`, just fetch the ones we need for this check.
# Without the GLOBAL modifier, SHOW VARIABLES displays the values that are used for
# the current connection to MariaDB.
sql = """
sql = '''
show global variables
where variable_name like 'innodb_buffer_pool_size'
or variable_name like 'innodb_log_file_size'
or variable_name like 'innodb_log_files_in_group'
or variable_name like 'innodb_redo_log_capacity'
;
"""
'''
return lib.base.coe(lib.db_mysql.select(conn, sql))


def main():
"""The main function. Hier spielt die Musik.
"""

# logic taken from mysqltuner.pl:mysql_innodb(), section # InnoDB Buffer Pool Size, v1.9.8
# logic taken from mysqltuner.pl:mysql_innodb(), section # InnoDB Buffer Pool Size, v2.2.12
# including variable names

# parse the command line, exit with UNKNOWN if it fails
Expand All @@ -116,51 +117,87 @@ def main():
conn = lib.base.coe(lib.db_mysql.connect(mysql_connection))
lib.base.coe(lib.db_mysql.check_select_privileges(conn))

myvar = lib.db_mysql.lod2dict(get_vars(conn))
if myvar.get('innodb_log_files_in_group', None) is None:
# innodb_log_files_in_group removed in MariaDB 10.6.0
myvar['innodb_log_files_in_group'] = '1'
engines = lib.db_mysql.get_engines(conn)

if not engines.get('have_innodb', ''):
lib.db_mysql.close(conn)
lib.base.cu('InnoDB Storage Engine not available.')
if engines['have_innodb'] != 'YES':
lib.db_mysql.close(conn)
lib.base.cu('InnoDB Storage Engine is disabled.')

sql = 'select sum(data_length+index_length) as size from information_schema.tables where table_schema not in ("information_schema", "performance_schema", "mysql") and engine = "innodb";'
myvar = lib.db_mysql.lod2dict(get_vars(conn))
if myvar.get('innodb_log_files_in_group', None) is None \
or myvar.get('innodb_log_files_in_group', 0) == 0:
# innodb_log_files_in_group removed in MariaDB 10.6.0
myvar['innodb_log_files_in_group'] = '1'

sql = '''
select sum(data_length+index_length) as InnoDB
from information_schema.tables
where
table_schema not in ("information_schema", "performance_schema", "mysql")
and engine = "innodb";
'''
enginestats = lib.base.coe(lib.db_mysql.select(conn, sql))
if enginestats[0]['size'] is None:
enginestats[0]['size'] = 0
if enginestats[0]['InnoDB'] is None:
enginestats[0]['InnoDB'] = 0

lib.db_mysql.close(conn)

# calculations
mycalc = {}
mycalc['innodb_log_size_pct'] = round(int(myvar['innodb_log_file_size']) * int(myvar['innodb_log_files_in_group']) / int(myvar['innodb_buffer_pool_size']) * 100, 1)
mycalc['innodb_log_size_pct'] = int(
int(myvar['innodb_log_file_size']) * int(myvar['innodb_log_files_in_group']) / \
int(myvar['innodb_buffer_pool_size']) * 100
)

# InnoDB Buffer Pool Size
msg += 'InnoDB buffer pool / data size: {} / {}'.format(
# Output changed compared to MySQLTuner (to be more clear).

# handling innodb_redo_log_capacity is specific to MySQL only - not implemented here
# MariaDB: https://mariadb.com/kb/en/innodb-redo-log/
# MySQL: https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_redo_log_capacity

if not lib.base.X86_64 and int(myvar['innodb_buffer_pool_size']) > 4294967295:
local_state = STATE_WARN
state = lib.base.get_worst(state, local_state)
msg += ('InnoDB Buffer Pool size ({}) limit reached for 32 bits architecture. '
'Limit innodb_buffer_pool_size under {}{}.').format(
lib.human.bytes2human(int(myvar['innodb_buffer_pool_size'])),
lib.human.bytes2human(4294967295),
lib.base.state2str(local_state),
)
if lib.base.X86_64 and int(myvar['innodb_buffer_pool_size']) > 18446744073709551615:
local_state = STATE_WARN
state = lib.base.get_worst(state, local_state)
msg += ('InnoDB Buffer Pool size ({}) limit reached for 64 bits architecture. '
'Limit innodb_buffer_pool_size under {}{}.').format(
lib.human.bytes2human(int(myvar['innodb_buffer_pool_size'])),
lib.human.bytes2human(18446744073709551615),
lib.base.state2str(local_state),
)

msg += 'Data size: {}, innodb_buffer_pool_size: {}'.format(
lib.human.bytes2human(int(enginestats[0]['InnoDB'])),
lib.human.bytes2human(int(myvar['innodb_buffer_pool_size'])),
lib.human.bytes2human(int(enginestats[0]['size'])),
)
if int(myvar['innodb_buffer_pool_size']) <= int(enginestats[0]['size']):
size_state = STATE_WARN
state = lib.base.get_worst(state, size_state)
msg += '{}. Set innodb_buffer_pool_size >= {}. '.format(
lib.base.state2str(size_state, prefix=' '),
lib.human.bytes2human(int(enginestats[0]['size'])),
if int(myvar['innodb_buffer_pool_size']) <= int(enginestats[0]['InnoDB']):
local_state = STATE_WARN
state = lib.base.get_worst(state, local_state)
msg += ' doesn\'t fit {}. Set innodb_buffer_pool_size >= {}. '.format(
lib.base.state2str(local_state),
lib.human.bytes2human(int(enginestats[0]['InnoDB'])),
)
else:
msg += '. '

msg += 'innodb_log_file_size * innodb_log_files_in_group / innodb_buffer_pool_size = {} * {} / {} = {}%'.format(
msg += ('\nRatio innodb_log_file_size ({}) * innodb_log_files_in_group ({}) '
'vs. innodb_buffer_pool_size ({}): {}% ').format(
lib.human.bytes2human(int(myvar['innodb_log_file_size'])),
myvar['innodb_log_files_in_group'],
lib.human.bytes2human(int(myvar['innodb_buffer_pool_size'])),
mycalc['innodb_log_size_pct']
mycalc['innodb_log_size_pct'],
)

# badprint
if int(mycalc['innodb_log_size_pct']) < 20 or int(mycalc['innodb_log_size_pct']) > 30:
ratio_state = STATE_WARN
state = lib.base.get_worst(state, ratio_state)
Expand All @@ -172,8 +209,6 @@ def main():
int(myvar['innodb_buffer_pool_size']) / int(myvar['innodb_log_files_in_group']) / 4
)
)
else:
msg += '.'

perfdata += lib.base.get_perfdata('mysql_innodb_buffer_pool_size', myvar['innodb_buffer_pool_size'], 'B', None, None, 0, None)
perfdata += lib.base.get_perfdata('mysql_innodb_log_file_size', myvar['innodb_log_file_size'], 'B', None, None, 0, None)
Expand Down

0 comments on commit ea04942

Please sign in to comment.