Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partition: add doc for global index #17979

Closed
wants to merge 13 commits into from
60 changes: 60 additions & 0 deletions partitioned-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -1662,6 +1662,66 @@ CREATE TABLE t (a varchar(20), b blob,
ERROR 1503 (HY000): A UNIQUE INDEX must include all columns in the table's partitioning function
```

#### Global indexes

If you need to create unique indexes that **do not include all the columns used in the partition expressions**, you can achieve this by enabling the [`tidb_enable_global_index`](/system-variables.md#tidb_enable_global_index-new-in-v760) system variable.

Defined2014 marked this conversation as resolved.
Show resolved Hide resolved
Previously an index on partitioned tables are created for each partition, which is the reason for the limitation that all unique keys need to include all partitioning columns. The uniqueness can only be enforced within each partition. A global index will be created on table level, so it can enforce uniqueness regardless of partitioning. Note that this has implications on partitioning management, `DROP`, `TRUNCATE`, and `REORGANIZE PARTITION` will also need to manage the table level global index.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Previously an index on partitioned tables are created for each partition, which is the reason for the limitation that all unique keys need to include all partitioning columns. The uniqueness can only be enforced within each partition. A global index will be created on table level, so it can enforce uniqueness regardless of partitioning. Note that this has implications on partitioning management, `DROP`, `TRUNCATE`, and `REORGANIZE PARTITION` will also need to manage the table level global index.
Previously an index on partitioned tables are created for each partition, which is the reason for the limitation that all unique keys need to include all partition keys. The uniqueness can only be enforced within each partition. A global index will be created on table level, so it can enforce uniqueness regardless of partitioning. Note that this has implications on partitioning management, `DROP`, `TRUNCATE`, and `REORGANIZE PARTITION` will also need to manage the table level global index.


After enabling this variable, any unique index that does not meet the preceding constraint will automatically become a global index.

Defined2014 marked this conversation as resolved.
Show resolved Hide resolved
```sql
SET tidb_enable_global_index = ON;

CREATE TABLE t1 (
col1 INT NOT NULL,
col2 DATE NOT NULL,
col3 INT NOT NULL,
col4 INT NOT NULL,
UNIQUE KEY uidx12(col1, col2),
UNIQUE KEY uidx3(col3)
)
PARTITION BY HASH(col3)
PARTITIONS 4;
```

In this example, the unique index `uidx12` will be implicitly a global index, but `uidx3` remains a regular unique index.

Note that a **clustered index** cannot be a global index, as shown in the following example:

Defined2014 marked this conversation as resolved.
Show resolved Hide resolved
```sql
SET tidb_enable_global_index = ON;

CREATE TABLE t1 (
col1 INT NOT NULL,
col2 DATE NOT NULL,
PRIMARY KEY (col2) clustered
) PARTITION BY HASH(col1) PARTITIONS 5;
```
Defined2014 marked this conversation as resolved.
Show resolved Hide resolved

```
ERROR 1503 (HY000): A CLUSTERED INDEX must include all columns in the table's partitioning function
```

mjonss marked this conversation as resolved.
Show resolved Hide resolved
The reason is that if the clustered index is a global index, the table will no longer be partitioned. This is because the key of the clustered index is also the record key, which means it should be on partition level. But the global index is on table level, which causes a conflict.

You can identify a global index by querying the [`information_schema.tidb_indexes`](/information-schema/information-schema-tidb-indexes.md) table and checking the table structure.

```sql
SELECT * FROM information_schema.tidb_indexes WHERE table_name='t1';
```

```
+--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
Defined2014 marked this conversation as resolved.
Show resolved Hide resolved
| TABLE_SCHEMA | TABLE_NAME | NON_UNIQUE | KEY_NAME | SEQ_IN_INDEX | COLUMN_NAME | SUB_PART | INDEX_COMMENT | Expression | INDEX_ID | IS_VISIBLE | CLUSTERED | IS_GLOBAL |
+--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
| test | t1 | 0 | uidx12 | 1 | col1 | NULL | | NULL | 1 | YES | NO | 1 |
| test | t1 | 0 | uidx12 | 2 | col2 | NULL | | NULL | 1 | YES | NO | 1 |
| test | t1 | 0 | uidx3 | 1 | col3 | NULL | | NULL | 2 | YES | NO | 0 |
+--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
3 rows in set (0.00 sec)
```

### Partitioning limitations relating to functions

Only the functions shown in the following list are allowed in partitioning expressions:
Expand Down
11 changes: 7 additions & 4 deletions placement-rules-in-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,9 @@ CREATE PLACEMENT POLICY storageforhisotrydata CONSTRAINTS="[+node=history]";
CREATE PLACEMENT POLICY storagefornewdata CONSTRAINTS="[+node=new]";
CREATE PLACEMENT POLICY companystandardpolicy CONSTRAINTS="";

CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE)
SET tidb_enable_global_index = ON;

CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE, UNIQUE INDEX idx(id))
PLACEMENT POLICY=companystandardpolicy
PARTITION BY RANGE( YEAR(purchased) ) (
PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY=storageforhisotrydata,
Expand All @@ -312,12 +314,13 @@ PARTITION BY RANGE( YEAR(purchased) ) (
);
```

If no placement policy is specified for a partition in a table, the partition attempts to inherit the policy (if any) from the table. In the preceding example:
If no placement policy is specified for a partition in a table, the partition attempts to inherit the policy (if any) from the table. If the table has a [global index](/partitioned-table.md#global-indexes), the index will apply the same placement policy as the table. In the preceding example:

- The `p0` partition will apply the `storageforhisotrydata` policy.
- The `p4` partition will apply the `storagefornewdata` policy.
- The `p1`, `p2`, and `p3` partitions will apply the `companystandardpolicy` placement policy inherited from the table `t1`.
- If no placement policy is specified for the table `t1`, the `p1`, `p2`, and `p3` partitions will inherit the database default policy or the global default policy.
- The global index `idx` will apply the same `companystandardpolicy` placement policy as the table `t1`.
- If no placement policy is specified for the table `t1`, then the `p1`, `p2`, and `p3` partitions and the global index `idx` will inherit the database default policy or the global default policy.

After placement policies are attached to these partitions, you can change the placement policy for a specific partition as in the following example:

Expand Down Expand Up @@ -479,4 +482,4 @@ After executing the statements in the example, TiDB will place the `app_order` d
| TiDB Lightning | Not compatible yet | An error is reported when TiDB Lightning imports backup data that contains placement policies |
| TiCDC | 6.0 | Ignores placement policies, and does not replicate the policies to the downstream |

</CustomContent>
</CustomContent>
6 changes: 5 additions & 1 deletion system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -2128,13 +2128,17 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;

### tidb_enable_global_index <span class="version-mark">New in v7.6.0</span>

> **Warning:**
>
> The feature controlled by this variable is an experimental feature. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub.

- Scope: SESSION | GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Boolean
- Default value: `OFF`
- Possible values: `OFF`, `ON`
- This variable controls whether to support creating `Global indexes` for partitioned tables. `Global index` is currently in the development stage. **It is not recommended to modify the value of this system variable**.
- This variable controls whether to support creating [`global indexes`](/partitioned-table.md#global-indexes) for partitioned tables. When this variable is enabled, TiDB creates unique indexes that do not contain all columns in the partition expressions.
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

### tidb_enable_non_prepared_plan_cache

Expand Down
Loading