Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL] Enable batching of explicit row lock requests by default #22519

Open
1 task done
pkj415 opened this issue May 23, 2024 · 0 comments
Open
1 task done

[YSQL] Enable batching of explicit row lock requests by default #22519

pkj415 opened this issue May 23, 2024 · 0 comments
Assignees
@d-uspenskiy
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@pkj415
Copy link
Contributor

pkj415 commented May 23, 2024 •
edited by jira bot
Loading

Jira Link: DB-11445

Description

09e46ba introduces batch execution of explicit row lock requests in Pg. This was disabled by default by setting yb_explicit_row_locking_batch_size to 1. Enable it by default.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@pkj415 pkj415 added area/ysql Yugabyte SQL (YSQL) status/awaiting-triage Issue awaiting triage labels May 23, 2024
@pkj415 pkj415 self-assigned this May 23, 2024
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels May 23, 2024
@pkj415 pkj415 added 2024.1 Backport Required and removed kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels May 23, 2024
@sushantrmishra sushantrmishra removed the status/awaiting-triage Issue awaiting triage label Aug 8, 2024
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Aug 21, 2024
d-uspenskiy added a commit that referenced this issue Sep 5, 2024
@d-uspenskiy
[#22519] YSQL: Simplify API of the ExplicitRowLockBuffer class
16262f7 
Summary:
This diff simplifies API of the `ExplicitRowLockBuffer` class by omitting `ybctid_reader` argument from `Add` and `Flush` methods.
The `ybctid_reader` is stored as an object field instead.

**Note:**
In context of this class several formatting changes are performed.
Jira: DB-11445

Test Plan: Jenkins

Reviewers: pjain, kramanathan, telgersma, myang

Reviewed By: kramanathan

Subscribers: ybase, yql

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D37467
jasonyb pushed a commit that referenced this issue Sep 6, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 94
1463ddd 
Summary:
 c587efd [docs] minor edit (#23796)
 31e09f3 [PLAT-15029] yba installer split data and software directory setup
 f5ba17d [PLAT-15039]: Fix bootstrap on bi-directional xCluster config creation
 578248a [#23770] YSQL: Deterministically populate catalog cache in tests with Connection Manager enabled
 7b1f22a [#23799] test: Fixed PgTableSizeTest.SharedTableSize test for pg15
 788434a [#18771, #21352] docdb: Fix LightweightMessage max size when parsing
 02ced43 [#22821] YSQL: Preserve local limit in a multi-page read
 50ff737 [#23741] docdb: Fix cloning of colocated databases with only parent table
 Excluded: 9889df7 [#23706] YSQL: Add table-level catcache Prometheus metrics
 c770d79 [#23747] MetaCache: Callback should not be called while holding the lock
 Excluded: 40689bc [#22150] YSQL, QueryDiagnostics:  EXPLAIN (ANALYZE, DIST) support for queryDiagnostics
 1655e69 [PLAT-15148]: Set XCluster Table Status to DroppedFromTarget if table in replication is dropped from target only
 16262f7 [#22519] YSQL: Simplify API of the ExplicitRowLockBuffer class
 6614afb [PLAT-14958][PLAT-14959] Make ssh fields optional if skipProvisioning is true
 1153b56 [PLAT-14867] Make sure restart alerts don't trigger for small time updates during NTP sync
 bf1c7bc [PLAT-12226] Add connection pooling status to universe health check
 38d8ae8 [PLAT-14805]Support adding EAR configs
 a180bef [#19134] YSQL, ASH: Setting ASH circular buffer size based on the number of cores
 7d8fc76 Adjust heading link (#23807)
 4c6cf5a [PLAT-4899]Basic validation of certificates
 f24eb10 [#23787] YSQL: Avoid executing conn mgr guc variables hooks for parallel workers
 ee18df8 [PLAT-13921] [K8] [UI] Universe action tasks are disabled after a failed shrink rr node task
 Excluded: dcf1821 [#23797] YSQL: Modify some tests to run in single connection mode with Connection Manager
 0ac22cd [Docs] Remove Drift chat bot (#23802)
 0e91003 [#22825] DocDB: Vector Index General Read Path with DummyANN
 e8f09b5 [PLAT-15175] Make runtime conf for skipping cluster consistency check public
 ee479ee Versionwarning (#23781)
 a05c6a3 [DB-12681] yugabyted-ui: Add Voyager commands to different Voyager phases in the UI.
 cc80d59 [#23777] yugabyted: updating the pg parity testcase to reflect the new gflags enabled for the pg parity feature.

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37822
d-uspenskiy added a commit that referenced this issue Sep 10, 2024
@d-uspenskiy
[#22519] YSQL: Move ExplicitRowLockBuffer class into separate file
eed826d 
Summary:
Move the `ExplicitRowLockBuffer` class into newly created file `pg_explicit_row_lock_buffer.cc`
Jira: DB-11445

Test Plan: Jenkins

Reviewers: pjain, kramanathan, myang, telgersma

Reviewed By: kramanathan

Subscribers: yql

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D37797
jasonyb pushed a commit that referenced this issue Sep 12, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 98
c368a58 
Summary:
 f5ad1fb [#23855] Fixing the calculation of automatic refactoring count.
 eed826d [#22519] YSQL: Move ExplicitRowLockBuffer class into separate file
 99a27e6 [PLAT-14522] Taking yba-ctl backups with prometheus HTTPS
 e2a84b0 [PLAT-15044] Add preflight check for node addition in provider
 7e40d89 [#20769] XCluster: Dynamically apply cdc_wal_retention_time_secs for XCluster
 aa41478 [#23858] build: fix ./yb_build.sh release --gcc11
 bcf7f47 [PLAT-13910] Improve IAM credentials fetch logging and add retries
 303a202 [#23778] xCluster: Remove the capability to rename xCluster replication groups
 Excluded: 58fd26e [#23652] YSQL: Fix TestPgRegressAnalyze.java timeout / database drop failure on TSAN build
 31da65b [doc] yb_enable_bitmapscan flag (#23854)
 798db14 [#20335] DocDB: Use MonoClock for write query metric
 80779d8 [#23860] xCluster: Add automatic ddl mode proto fields
 afd763d Revert "Revert "[PLAT-14786] Add support to node_agent install to use bind ip and node_external_fqdn""
 e86951a [#23841] docdb: Disable stack trace tracking in TSAN builds
 d600608 [PLAT-15244] Fix schedule not getting updated on edit schedule API call
 3c0df09 [PLAT-15214][PLAT-15232]YBC version upgrade to 2.2.0.0-b6 and enable YBC verbose by default
 aa7372e [#23478] YSQL: fix connection manager session variable case sensitivity issue
 0d53558 [PLAT-14810][PLAT-14811][YBA CLI] Support adding and editing EIT configurations
 ffa537e [PLAT-10706][dr] Support retry-ability of failover and switchover
 Excluded: 5ae4558 [#23578] YSQL: Add HELP and TYPE to :13000/prometheus-metrics
 3aa7459 [PLAT-15016] Handle gflag_group changes for ENHANCED_PG_COMPATIBILITY group in 2024.1.3
 c89356c [PLAT-10592][YBA] Changes to support global tserver/master service in K8s
 7fc3b76 [#3893] YCQL: Align 2 system_schema.* tables with Cassandra
 da6274e [23646] Test Stability: Fix PgMiniTest.FollowerReads
 5fa6dc9 [PLAT-15180][Platform][UI][PITR]Create Restore Backup modal
 be0d1d1 [PLAT-15247][Platform][Backup]Create Backup scheduled policy List

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37981
d-uspenskiy added a commit that referenced this issue Sep 13, 2024
@d-uspenskiy
[#22519] YSQL: Substitute YBSetRowLockPolicy function with YBGetDocDB…
7f59bd8 
…WaitPolicy

Summary:
Auxiliary diff to fix `TODO` section to simplify the `YBSetRowLockPolicy` function usage by substituting it with the `YBGetDocDBWaitPolicy` function.

The purpose of this diff is to reduce changes in main diff for the #22519 issue (https://phorge.dev.yugabyte.com/D37821)
Jira: DB-11445

Test Plan: Jenkins

Reviewers: kramanathan, tfoucher

Reviewed By: tfoucher

Subscribers: yql

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D38003
jasonyb pushed a commit that referenced this issue Sep 16, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 100
1a5c22b 
Summary:
 32572f7 [#23629]yugabyted: 2nd node doesn't join if webserver_port is changed
 b2d16eb [PLAT-15182]: Reset config working incorrectly for kubernetes universe
 dd60793 [#23843] YSQL: Do not retry schema mismatch errors that occur as part of a batched execution.
 22b6a47 [#23661] YSQL: Change the default wal_level to logical
 5eb925b [#23410] YSQL: Fix CREATE TABLE with pg_default tablespace in colocated databases
 2c0c0eb [#23913] build: fix ysql conn mgr ASAN issue
 b98984a Disable failing DR local provider tests
 48e3c40 Update YBA HA pre-reqs (#23865)
 9baa0be [docs] Release notes for 2.20.2.4 (#23880)
 a2a74d0 [Docs][Voyager] Added a known-issue in PG and some missing entries from PG compatibility doc (#23906)
 0a738ee [#23336] YSQL: Fix tablespace id update in pg_yb_tablegroup after ALTER TABLE CASCADE
 a0e2c4a [PLAT-15156] Pass pg_max_mem_mb during configure phase
 Excluded: 7f59bd8 [#22519] YSQL: Substitute YBSetRowLockPolicy function with YBGetDocDBWaitPolicy

MERGE:

- yb_pg_publication.out:
  - CREATE PUBLICATION testpub_foralltables FOR ALL TABLES;: YB master
    22b6a47 causes WARNING about
    wal_level not being "logical" to go away.  This warning is only
    present in YB pg15 because YB pg15
    5c5a2a0 was not able to suppress
    this WARNING for this one case (see that commit message for more
    details).
- yb_publication.out:
  - CREATE PUBLICATION testpub FOR ALL TABLES;: (same)
- yb_pg_pg_stat_statements.out
  - Check WAL is generated for the above statements: "DROP TABLE
    pgss_test" row's wal column values change.  Likely related to YB
    master 22b6a47.  Not sure how
    exactly it relates, but since it makes the test closer to the
    original pg_stat_statements.out, take it without question.
- yb_colocated_tables_with_tablespaces.{sql,out}:
  - Explicitly specifying pg_default tablespace must not create a new
    tablegroup: YB master 5eb925b adds
    these queries with SELECT * from system table, and upstream PG
    changes that to include the oid column.  We don't want to show the
    oid column since that can easily change, so replace * with explicit
    column names.

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: tfoucher, jenkins-bot, stiwary, utkarsh.munjal

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D38035
d-uspenskiy added a commit that referenced this issue Sep 17, 2024
@d-uspenskiy
[BACKPORT pg15-cherrypicks][#22519] YSQL: Substitute YBSetRowLockPoli…
0f0c36f 
…cy function with YBGetDocDBWaitPolicy

Summary:
Original commit: 7f59bd8 / D38003
Auxiliary diff to fix `TODO` section to simplify the `YBSetRowLockPolicy` function usage by substituting it with the `YBGetDocDBWaitPolicy` function.

The purpose of this diff is to reduce changes in main diff for the #22519 issue (https://phorge.dev.yugabyte.com/D37821)
Jira: DB-11445

Merge:
  src/postgres/src/backend/executor/nodeYbSeqscan.c - trivial conflict between YB pg15 71dc8d0 and YB master 7f59bd8 in the YbSeqNext function

Test Plan: Jenkins

Reviewers: jason, tfoucher

Reviewed By: jason

Subscribers: yql

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D38071
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@d-uspenskiy d-uspenskiy

Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
Wait-Queue Based Locking
Status: In Progress
Milestone
No milestone
Development

No branches or pull requests
4 participants
@sushantrmishra @pkj415 @yugabyte-ci @d-uspenskiy