Simultaneous incremental scans interfere with each other causing replication to not proceed

[liuzix]

Basic Scenario

1. grpc stream 1 requests region 1.
2. grpc stream 1 starts incremental scan on region 1.
3. grpc stream 2 requests region 1.
4. grpc stream 2 starts incremental scan on region 1.
5. grpc stream 1 aborts before the scan on region 1 was finished.
6. grpc stream 2 finishes scanning region 1.
7. grpc stream 2's resolved ts on region 1 fails to proceed.

Cause

TiKV only lets stream 1 track the locks and builds the resolver, which is responsible for calculating the resolved-ts. Since all connections to the same region shares the resolver, failure to build resolver prevents all other connections from receiving resolved-ts.

Implications

Users having multiple changefeeds on the same set of tables are likely to experience replication interruption or delay. The problem manifests itself as both resolved-ts and checkpoint-ts not progressing.

[amyangfei]

This has been fixed