Skip to content

Latest commit

 

History

History
51 lines (40 loc) · 2.75 KB

upstream-downstream-diff.md

File metadata and controls

51 lines (40 loc) · 2.75 KB
Raw
title summary aliases
Data Check for TiDB Upstream and Downstream Clusters
Learn how to check data for TiDB upstream and downstream clusters.
/docs/dev/sync-diff-inspector/upstream-downstream-diff/
/docs/dev/reference/tools/sync-diff-inspector/tidb-diff/

Data Check for TiDB Upstream and Downstream Clusters

You can use TiDB Binlog to build upstream and downstream clusters of TiDB. When Drainer replicates data to TiDB, the checkpoint is saved and the TSO mapping relationship between the upstream and downstream is also saved as ts-map. To check data between the upstream and downstream, configure snapshot in sync-diff-inspector.

Step 1: obtain ts-map

To obtain ts-map, execute the following SQL statement in the downstream TiDB cluster:

mysql> select * from tidb_binlog.checkpoint;
+---------------------+---------------------------------------------------------------------------------------------------------+
| clusterID           | checkPoint                                                                                              |
+---------------------+---------------------------------------------------------------------------------------------------------+
| 6711243465327639221 | {"commitTS":409622383615541249,"ts-map":{"primary-ts":409621863377928194,"secondary-ts":409621863377928345}} |
+---------------------+---------------------------------------------------------------------------------------------------------+

Step 2: configure snapshot

Then configure the snapshot information of the upstream and downstream databases by using the ts-map information obtained in Step 1.

Here is a configuration example of the Datasource config section:

######################### Datasource config ########################
[data-sources.uptidb]
    host = "172.16.0.1"
    port = 4000
    user = "root"
    password = ""
    snapshot = "409621863377928194"

[data-sources.downtidb]
    host = "172.16.0.2"
    port = 4000
    user = "root"
    snapshot = "409621863377928345"

Note:

  • Set db-type of Drainer to tidb to ensure that ts-map is saved in the checkpoint.
  • Modify the Garbage Collection (GC) time of TiKV to ensure that the historical data corresponding to snapshot is not collected by GC during the data check. It is recommended that you modify the GC time to 1 hour and recover the setting after the check.
  • In some versions of TiDB Binlog, master-ts and slave-ts are stored in ts-map. master-ts is equivalent to primary-ts and slave-ts is equivalent to secondary-ts.
  • The above example only shows the section of Datasource config. For complete configuration, refer to sync-diff-inspector User Guide.