Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL] pg_partman: Make partition creation idempotent #23163

Closed
yugabyte-ci opened this issue Jul 10, 2024 · 0 comments
Closed

[YSQL] pg_partman: Make partition creation idempotent #23163

yugabyte-ci opened this issue Jul 10, 2024 · 0 comments
Assignees
@devansh-ism
Labels
jira-originated kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@yugabyte-ci
Copy link
Contributor

yugabyte-ci commented Jul 10, 2024 •
edited by jira bot
Loading

Jira Link: DB-12101
@yugabyte-ci yugabyte-ci added jira-originated kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Jul 10, 2024
devansh-ism pushed a commit that referenced this issue Jul 23, 2024
[#23163] YSQL: pg_partman: make 'inherit_template_properties' idempotent
9e046fb 
Summary:
Currently transactional DDL is not supported in YugabyteDB . Due to which if any procedure or function which is performing multiple DDLs in a
transactional context can lead to issue of some DDLs getting executed and committed even if stored procedure failed due to some exception
while running stored procedure or Postgres backend process or tserver process kill.

This diff makes `inherit_template_properties` function idempotent such that multiple calls to don't have any additional consequences.

- This function is used to inherit the properties of the template table to newly created child tables.
- For PG11, it is used to inherit non-partition-key unique indexes & primary keys.
- Changes Done
  - Before creating a new index on the child table check if that index is already present for the child table to avoid duplicate index creation.
Jira: DB-12101

Test Plan: jenkins: compile only

Reviewers: skumar, jason, hsunder

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D36676
devansh-ism pushed a commit that referenced this issue Jul 23, 2024
[#23163] YSQL: pg_partman: make 'apply_publications' idempotent
2248dcd 
Summary:
Currently transactional DDL is not supported in YugabyteDB . Due to which if any procedure or function which is performing multiple DDLs in a
transactional context can lead to issue of some DDLs getting executed and committed even if stored procedure failed due to some exception
while running stored procedure or Postgres backend process or tserver process kill.

This diff makes apply_publications.sql function idempotent such that multiple calls to don't have any additional consequences.

 - This function is used to add child tables to publication
 - Changes done
   - Added a check to not invoke ALTER PUBLICATION <publication> ADD table <child_table> if child_table is already part of the publication.
Jira: DB-12101

Test Plan: jenkins: compile only

Reviewers: skumar, jason, hsunder

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D36677
devansh-ism pushed a commit that referenced this issue Jul 23, 2024
[#23163] YSQL: pg_partman: Make partition creation idempotent
7b32d05 
Summary:
Currently transactional DDL is not supported in YugabyteDB . Due to which if any procedure or function which are performing multiple DDLs in a
transactional context can lead to issue of DDLs getting executed and committed even if stored procedure failed due to some exception while running stored procedure or
ungraceful Postgres backend process or tserver process kill.

pg_partman extension uses `create_parent` function as main function to create child partitions of a partitioned table. `create_parent` itself calls other pg_partman functions that performs DDLs
to create and attach partitioned tables and add some other objects like indexes, constraints and adding privileges. Hence it required to make complete workflow of `create_parent` idempotent.

create_parent.sql

- Base function called for creating child partitions for a partitioned table.
- Changes done
  - Make default table creation idempotent with not-exist addition
  - Check before attaching default table if it is already attached

create_partition_time/id.sql

 - Functions called by create_parent and run_maintenance. Creation and attaching of child tables except for the default child table occurs here. the default child is handled at create_parent only.
 - Changes done
  - Get list of child tables attached to parent table into v_child_tables
  - Loop over the partition names for which want to create and attach new partition, name it is partition_name
    - If partition_name is present in v_child_tables, then continue
    - If partition_name is not attached but created, then skip creation -> have to test this -> made changes.
Jira: DB-12101

Test Plan: jenkins: compile only

Reviewers: skumar, hsunder, jason

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D36428
jasonyb pushed a commit that referenced this issue Jul 23, 2024
@jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 60+61
d517b7e 
Summary:
5a76f6a [#23179] CDCSDK: Support data types with dynamically alloted oids in CDC
5820ccd [PLAT-14710] Do not return apiToken in response to getSessionInfo
dcfa9cd [docs] updates to CVE table status column (#23225)
c0b1ee8 [docs] Fix load balance keyword in drivers page (#23253)
2becb46 [docs] Add basic troubleshooting steps for read restart errors. (#23228)
1b9be2e [PLAT-12733] Kubernetes overrides in v2 api
50422f8 [#23011] YSQL: Enable ALTER TABLE IF EXISTS t RENAME c1 TO c2
7c55b95 [PLAT-14073] DB scoped failover+repair
5ac65eb [#22449] YSQL: import wal2json wal2json_2_6
9e046fb [#23163] YSQL: pg_partman: make 'inherit_template_properties' idempotent
2248dcd [#23163] YSQL: pg_partman: make 'apply_publications' idempotent
7b32d05 [#23163] YSQL: pg_partman: Make partition creation idempotent
b0349fe [PLAT-14710] update api doc for /session_info
fe37ffd [#23240] CDCSDK: Make test replicationConnectionConsumptionAllDataTypes more resilient

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: tfoucher

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D36789
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@devansh-ism devansh-ism

Labels
jira-originated kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yugabyte-ci @devansh-ism