Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[BACKPORT pg15-cherrypicks][#23706] YSQL: Add table-level catcache Pr…
…ometheus metrics Summary: Original commit: 9889df7 / D37633 PG15 removes the YB pinned objects in bea1ffb. YB master 9889df7 conflicts with this (false conflict, we keep the PG15 code here). There is also the following logical conflict: PG15 adds 6 new catcaches: - 2 on pg_parameter_acl - 2 on pg_publication_namespace - 1 on pg_range - 1 on pg_statistic_ext_data Of these, pg_parameter_acl, pg_publication_namespace, and pg_statistic_ext_data are new catalog tables introduced in pg15. Adjust catalog cache metrics to account for new catalog tables/caches introduced in pg15, adapt according to YB pg15 merge 602cb2d. - src/postgres/src/backend/utils/cache/syscache.h: - enum YbCatalogCacheTable: - YB master 9889df7 adds this enum representing the catalog tables that have catcaches. - PG15 adds three new catalog tables. - Resolution: add the three new catalog tables to this enum. - src/postgres/src/backend/utils/cache/syscache.c: - `static const char *yb_cache_table_name_table` / `static YbCatalogCacheTable yb_catalog_cache_tables`: - YB master 9889df7 adds this list of all the PG catalog tables there are catcaches for. - PG15 adds three new catalog tables and six new catcaches. - Resolution: add the six entries corresponding to the new catcaches to these lists. - src/postgres/yb-extensions/yb_pg_metrics/yb_pg_metrics.c: - enum statementType: - YB master 9889df7 adds one CatCacheTableMisses entry per table that has a catcache (total 47 entries). - PG15 adds three new catalog tables. - Resolution: add the three new CatCacheTableMisses entries corresponding to the new catcaches to this enum (total 50 entries). - src/yb/yql/pgwrapper/pg_libpq-test.cc: - GetCatalogTableNameFromIndexName: - YB master commit 9889df7 adds a test that has a list of all the PG catalog tables. - PG15 adds three new catalog tables. - Resolution: add the three new catalog tables to this list. Conflict is related to c271944. - `src/postgres/yb-extensions/yb_pg_metrics/yb_pg_metrics.c`: - In c271944, are correcting the `enum statementType`'s `CatCacheIdMisses` section so that the end-of-section market points to the last enum in the section rather than being its own enum after the end of the section. In this revision, we are adding a new section, `CatCacheTableMisses`, to `enum statementType`, so the changes are unrelated. We make sure, however, that the end-of-section marker here points to the last enum in the section to avoid the issue addressed in c271944. Adds table-level Prometheus metrics for the number of catcache misses in addition to the current index-level ones added in D35792. These metrics are node-level, i.e. they will include the cache misses from all backends running on this node, but not the other nodes. They accessible via the `:13000/prometheus-metrics` endpoint. Including the metrics added in this revision, we now have three YSQL metrics that track catalog cache misses: - `CatalogCacheMisses` (no `table_name` field): Total number of cache misses across all catcaches. - `CatalogCacheMisses` (`table_name` field specified): Number of cache misses for the index specified by `table_name`. - `CatalogCacheTableMisses` (**new**): Number of cache misses for all catcaches on the table specified by `table_name`. Jira: DB-12616 Test Plan: ``` ./yb_build.sh --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.CatalogCacheIdMissMetricsTest ``` Manual test: ``` bin/yb-ctl create bin/ysqlsh yugabyte=# create table test(a int); CREATE TABLE yugabyte=# \c You are now connected to database "yugabyte" as user "yugabyte". yugabyte=# select * from t; a --- (0 rows) yugabyte=# \q wget 'http://127.0.0.1:13000/prometheus-metrics?reset_histograms=false&show_help=false' -O - 2>/dev/null | grep CatalogCacheTableMisses | grep count handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_aggregate",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_am",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_amop",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_amproc",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_attribute",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_auth_members",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_authid",exported_instance="kf-mbp-v47vl:9000"} 2 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_cast",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_class",exported_instance="kf-mbp-v47vl:9000"} 3 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_collation",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_constraint",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_conversion",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_database",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_default_acl",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_enum",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_event_trigger",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_foreign_data_wrapper",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_foreign_server",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_foreign_table",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_index",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_language",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_namespace",exported_instance="kf-mbp-v47vl:9000"} 2 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_opclass",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_operator",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_opfamily",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_partitioned_table",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_proc",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_publication",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_publication_rel",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_range",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_replication_origin",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_rewrite",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_sequence",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_statistic",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_statistic_ext",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_subscription",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_subscription_rel",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_tablespace",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_transform",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_ts_config",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_ts_config_map",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_ts_dict",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_ts_parser",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_ts_template",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_type",exported_instance="kf-mbp-v47vl:9000"} 1 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_user_mapping",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 handler_latency_yb_ysqlserver_SQLProcessor_CatalogCacheTableMisses_count{metric_id="yb.ysqlserver",metric_type="server",table_name="pg_yb_tablegroup",exported_instance="kf-mbp-v47vl:9000"} 0 1724889458674 ``` Reviewers: jason, tfoucher Reviewed By: jason Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D37871
- Loading branch information