Skip to content

Commit

Permalink
[docs] Fix tablet splitting flag default values and add max_create_ta…
Browse files Browse the repository at this point in the history
…blets_per_ts

Fix tablet splitting flag defaults and add max_create_tablets_per_ts
  • Loading branch information
SrivastavaAnubhav committed Jul 10, 2024
1 parent 56ee972 commit 9de2593
Show file tree
Hide file tree
Showing 11 changed files with 75 additions and 36 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ At creation time, you can presplit a table into the desired number of tablets. Y

To presplit a table into a desired number of tablets, you need the start key and end key for each tablet. This makes presplitting slightly different for hash-sharded and range-sharded tables.

The maximum number of tablets allowed at table creation time is controlled by [`max_create_tablets_per_ts`](../../../reference/configuration/yb-master/#max-create-tablets-per-ts). This also limits the number of tablets that can be created by tablet splitting.

#### Hash-sharded tables

Because hash sharding works by applying a hash function on all or a subset of the primary key columns, the byte space of the hash sharded keys is known ahead of time. For example, if you use a 2-byte hash, the byte space would be `[0x0000, 0xFFFF]`, as per the following illustration:
Expand Down Expand Up @@ -233,15 +235,15 @@ Automatic tablet splitting happens in three phases, determined by the shard coun

##### Low phase

In the low phase, each node has fewer than [`tablet_split_low_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-low-phase-shard-count-per-node) shards (8 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_low_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-low-phase-size-threshold-bytes) (512 MB by default).
In the low phase, each node has fewer than [`tablet_split_low_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-low-phase-shard-count-per-node) shards (1 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_low_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-low-phase-size-threshold-bytes) (128 MiB by default).

##### High phase

In the high phase, each node has fewer than [`tablet_split_high_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-high-phase-shard-count-per-node) shards (24 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_high_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-high-phase-size-threshold-bytes) (10 GB by default).
In the high phase, each node has fewer than [`tablet_split_high_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-high-phase-shard-count-per-node) shards (24 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_high_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-high-phase-size-threshold-bytes) (10 GiB by default).

##### Final phase

When the shard count exceeds the high phase count (determined by `tablet_split_high_phase_shard_count_per_node`, 24 by default), YugabyteDB splits tablets larger than [`tablet_force_split_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-force-split-threshold-bytes) (100 GB by default). This will continue until the [`tablet_split_limit_per_table`](../../../reference/configuration/yb-master/#tablet-split-limit-per-table) tablets per table limit is reached (256 tablets by default; if set to 0, there will be no limit).
Once the shard count exceeds the high phase count (determined by `tablet_split_high_phase_shard_count_per_node`, 24 by default), YugabyteDB splits tablets larger than [`tablet_force_split_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-force-split-threshold-bytes) (100 GiB by default). The maximum number of tablets is still limited by [`max_create_tablets_per_ts`](../../../reference/configuration/yb-master/#max-create-tablets-per-ts).

#### Post-split compactions

Expand Down
14 changes: 10 additions & 4 deletions docs/content/preview/reference/configuration/yb-master.md
Original file line number Diff line number Diff line change
Expand Up @@ -635,6 +635,12 @@ Default: 1024 * (7/10) (corresponding to an overhead of roughly 700 KiB per tabl

## Tablet splitting flags

##### --max_create_tablets_per_ts

The maximum number of tablets per tablet server that can be specified when creating a table. This also limits the number of tablets that can be created by tablet splitting.

Default: `50`

##### --enable_automatic_tablet_splitting

Enables YugabyteDB to [automatically split tablets](../../../architecture/docdb-sharding/tablet-splitting/#automatic-tablet-splitting), based on the specified tablet threshold sizes configured below.
Expand All @@ -657,7 +663,7 @@ Default: `1`

The size threshold used to determine if a tablet should be split when the tablet's table is in the "low" phase of automatic tablet splitting. See [`--tablet_split_low_phase_shard_count_per_node`](./#tablet-split-low-phase-shard-count-per-node).

Default: `128_MB`
Default: `128 MiB`

##### --tablet_split_high_phase_shard_count_per_node

Expand All @@ -669,19 +675,19 @@ Default: `24`

The size threshold used to determine if a tablet should be split when the tablet's table is in the "high" phase of automatic tablet splitting. See [`--tablet_split_high_phase_shard_count_per_node`](./#tablet-split-low-phase-shard-count-per-node).

Default: `10_GB`
Default: `10 GiB`

##### --tablet_force_split_threshold_bytes

The size threshold used to determine if a tablet should be split even if the table's number of shards puts it past the "high phase".

Default: `100_GB`
Default: `100 GiB`

##### --tablet_split_limit_per_table

The maximum number of tablets per table for tablet splitting. Limitation is disabled if this value is set to 0.

Default: `256`
Default: `0`

##### --index_backfill_tablet_split_completion_timeout_sec

Expand Down
1 change: 0 additions & 1 deletion docs/content/preview/releases/ybdb-releases/v2.14.md
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,6 @@ docker pull yugabytedb/yugabyte:2.14.11.0-b35
* [[17500](https://github.com/yugabyte/yugabyte-db/issues/17500)] [DocDB] Fix test YbAdminSnapshotSchedule.PgsqlCreateTable in case of colocated DBs
* [[17574](https://github.com/yugabyte/yugabyte-db/issues/17574)] [DocDB] Add EncryptedWritableFile::Size() to fix incorrect WAL footer start offset
* [[17592](https://github.com/yugabyte/yugabyte-db/issues/17592)] [[17760](https://github.com/yugabyte/yugabyte-db/issues/17760)] [DocDB] Rename WAL file after writing the file header on the disk
* [[17603](https://github.com/yugabyte/yugabyte-db/issues/17603)] [DocDB] Tablet splitting should respect FLAGS_max_create_tablets_per_ts.
* [[17663](https://github.com/yugabyte/yugabyte-db/issues/17663)] [DocDB] Clear drive load / leader load maps before appending to them
* [[17712](https://github.com/yugabyte/yugabyte-db/issues/17712)] [DocDB] Fix system partitions query to not return empty namespace name when its not set at table level
* [[17726](https://github.com/yugabyte/yugabyte-db/issues/17726)] [DocDB] A workaround for memory leak from InboundCall weak pointers
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ At creation time, you can presplit a table into the desired number of tablets. Y

To presplit a table into a desired number of tablets, you need the start key and end key for each tablet. This makes presplitting slightly different for hash-sharded and range-sharded tables.

The maximum number of tablets allowed at table creation time is controlled by [`max_create_tablets_per_ts`](../../../reference/configuration/yb-master/#max-create-tablets-per-ts). This also limits the number of tablets that can be created by tablet splitting.

#### Hash-sharded tables

Because hash sharding works by applying a hash function on all or a subset of the primary key columns, the byte space of the hash sharded keys is known ahead of time. For example, if you use a 2-byte hash, the byte space would be `[0x0000, 0xFFFF]`, as per the following illustration:
Expand Down Expand Up @@ -233,15 +235,15 @@ Automatic tablet splitting happens in three phases, determined by the shard coun

##### Low phase

In the low phase, each node has fewer than [`tablet_split_low_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-low-phase-shard-count-per-node) shards (8 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_low_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-low-phase-size-threshold-bytes) (512 MB by default).
In the low phase, each node has fewer than [`tablet_split_low_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-low-phase-shard-count-per-node) shards (1 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_low_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-low-phase-size-threshold-bytes) (128 MiB by default).

##### High phase

In the high phase, each node has fewer than [`tablet_split_high_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-high-phase-shard-count-per-node) shards (24 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_high_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-high-phase-size-threshold-bytes) (10 GB by default).
In the high phase, each node has fewer than [`tablet_split_high_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-high-phase-shard-count-per-node) shards (24 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_high_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-high-phase-size-threshold-bytes) (10 GiB by default).

##### Final phase

When the shard count exceeds the high phase count (determined by `tablet_split_high_phase_shard_count_per_node`, 24 by default), YugabyteDB splits tablets larger than [`tablet_force_split_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-force-split-threshold-bytes) (100 GB by default). This will continue until the [`tablet_split_limit_per_table`](../../../reference/configuration/yb-master/#tablet-split-limit-per-table) tablets per table limit is reached (256 tablets by default; if set to 0, there will be no limit).
When the shard count exceeds the high phase count (determined by `tablet_split_high_phase_shard_count_per_node`, 24 by default), YugabyteDB splits tablets larger than [`tablet_force_split_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-force-split-threshold-bytes) (100 GiB by default). The maximum number of tablets is still limited by [`max_create_tablets_per_ts`](../../../reference/configuration/yb-master/#max-create-tablets-per-ts).

#### Post-split compactions

Expand Down
14 changes: 10 additions & 4 deletions docs/content/stable/reference/configuration/yb-master.md
Original file line number Diff line number Diff line change
Expand Up @@ -635,6 +635,12 @@ Default: 1024 * (7/10) (corresponding to an overhead of roughly 700 KiB per tabl

## Tablet splitting flags

##### --max_create_tablets_per_ts

The maximum number of tablets per tablet server that can be specified when creating a table. This also limits the number of tablets that can be created by tablet splitting.

Default: `50`

##### --enable_automatic_tablet_splitting

Enables YugabyteDB to [automatically split tablets](../../../architecture/docdb-sharding/tablet-splitting/#automatic-tablet-splitting), based on the specified tablet threshold sizes configured below.
Expand All @@ -657,7 +663,7 @@ Default: `1`

The size threshold used to determine if a tablet should be split when the tablet's table is in the "low" phase of automatic tablet splitting. See [`--tablet_split_low_phase_shard_count_per_node`](./#tablet-split-low-phase-shard-count-per-node).

Default: `128_MB`
Default: `128 MiB`

##### --tablet_split_high_phase_shard_count_per_node

Expand All @@ -669,19 +675,19 @@ Default: `24`

The size threshold used to determine if a tablet should be split when the tablet's table is in the "high" phase of automatic tablet splitting. See [`--tablet_split_high_phase_shard_count_per_node`](./#tablet-split-low-phase-shard-count-per-node).

Default: `10_GB`
Default: `10 GiB`

##### --tablet_force_split_threshold_bytes

The size threshold used to determine if a tablet should be split even if the table's number of shards puts it past the "high phase".

Default: `100_GB`
Default: `100 GiB`

##### --tablet_split_limit_per_table

The maximum number of tablets per table for tablet splitting. Limitation is disabled if this value is set to 0.

Default: `256`
Default: `0`

##### --index_backfill_tablet_split_completion_timeout_sec

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ At creation time, you can presplit a table into the desired number of tablets. Y

In order to presplit a table into a desired number of tablets, you need the start key and end key for each tablet. This makes presplitting slightly different for hash-sharded and range-sharded tables.

The maximum number of tablets allowed at table creation time is controlled by [`max_create_tablets_per_ts`](../../../reference/configuration/yb-master/#max-create-tablets-per-ts). This also limits the number of tablets that can be created by tablet splitting.

### Hash-sharded tables

Because hash sharding works by applying a hash function on all or a subset of the primary key columns, the byte space of the hash sharded keys is known ahead of time. For example, if you use a 2-byte hash, the byte space would be `[0x0000, 0xFFFF]`.
Expand Down Expand Up @@ -237,15 +239,15 @@ Automatic tablet splitting happens in three phases, determined by the shard coun

#### Low Phase

In the low phase, each node has fewer than [`tablet_split_low_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-low-phase-shard-count-per-node) shards (8 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_low_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-low-phase-size-threshold-bytes) (512 MB by default).
In the low phase, each node has fewer than [`tablet_split_low_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-low-phase-shard-count-per-node) shards (8 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_low_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-low-phase-size-threshold-bytes) (512 MiB by default).

#### High Phase

In the high phase, each node has fewer than [`tablet_split_high_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-high-phase-shard-count-per-node) shards (24 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_high_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-high-phase-size-threshold-bytes) (10 GB by default).
In the high phase, each node has fewer than [`tablet_split_high_phase_shard_count_per_node`](../../../reference/configuration/yb-master/#tablet-split-high-phase-shard-count-per-node) shards (24 by default). In this phase, YugabyteDB splits tablets larger than [`tablet_split_high_phase_size_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-split-high-phase-size-threshold-bytes) (10 GiB by default).

#### Final Phase

Once the shard count exceeds the high phase count (determined by `tablet_split_high_phase_shard_count_per_node`, 24 by default), YugabyteDB splits tablets larger than [`tablet_force_split_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-force-split-threshold-bytes) (100 GB by default). This will continue until we reach the [`tablet_split_limit_per_table`](../../../reference/configuration/yb-master/#tablet-split-limit-per-table) tablets per table limit (256 tablets by default; if set to 0, there will be no limit).
Once the shard count exceeds the high phase count (determined by `tablet_split_high_phase_shard_count_per_node`, 24 by default), YugabyteDB splits tablets larger than [`tablet_force_split_threshold_bytes`](../../../reference/configuration/yb-master/#tablet-force-split-threshold-bytes) (100 GiB by default). The maximum number of tablets is still limited by [`max_create_tablets_per_ts`](../../../reference/configuration/yb-master/#max-create-tablets-per-ts).

### Post-split compactions

Expand Down
14 changes: 10 additions & 4 deletions docs/content/v2.14/reference/configuration/yb-master.md
Original file line number Diff line number Diff line change
Expand Up @@ -497,6 +497,12 @@ Clusters created using `yugabyted` always use a default value of `1`.

### Tablet splitting flags

##### --max_create_tablets_per_ts

The maximum number of tablets per tablet server that can be specified when creating a table. This also limits the number of tablets that can be created by tablet splitting.

Default: `50`

##### --enable_automatic_tablet_splitting

Enables YugabyteDB to [automatically split tablets](../../../architecture/docdb-sharding/tablet-splitting/#automatic-tablet-splitting), based on the specified tablet threshold sizes configured below.
Expand All @@ -519,7 +525,7 @@ Default: `8`

The size threshold used to determine if a tablet should be split when the tablet's table is in the "low" phase of automatic tablet splitting. See [`--tablet_split_low_phase_shard_count_per_node`](./#tablet-split-low-phase-shard-count-per-node).

Default: `512_MB`
Default: `512 MiB`

##### --tablet_split_high_phase_shard_count_per_node

Expand All @@ -531,19 +537,19 @@ Default: `24`

The size threshold used to determine if a tablet should be split when the tablet's table is in the "high" phase of automatic tablet splitting. See [`--tablet_split_high_phase_shard_count_per_node`](./#tablet-split-low-phase-shard-count-per-node).

Default: `10_GB`
Default: `10 GiB`

##### --tablet_force_split_threshold_bytes

The size threshold used to determine if a tablet should be split even if the table's number of shards puts it past the "high phase".

Default: `100_GB`
Default: `100 GiB`

##### --tablet_split_limit_per_table

The maximum number of tablets per table for tablet splitting. Limitation is disabled if this value is set to 0.

Default: `256`
Default: `0`

##### --index_backfill_tablet_split_completion_timeout_sec

Expand Down
Loading

0 comments on commit 9de2593

Please sign in to comment.