pd config can't be auto resumed if lightning panic

[glorv]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1. Run tidb-lightning with local backend
2. After tidb-lightning start running and pd-scheduler is patched, stop tidb-lightning process will kill -9

2. What did you expect to see? (Required)

The scheduler config should be restored to the origin value after 5min.

3. What did you see instead (Required)

The scheduler remained the changed value.

4. What is your TiDB version? (Required)

nightly

[glorv]

/cc @3pointer

[3pointer]

I have a draft PR to fix this issue. #31725. but cannot verify in tests. because we cannot make sure the ttl works through api.

// update config with ttl
curl http://{PD_IP}/pd/api/v1/config?ttlSecond=100 -XPOST -d '{"schedule.max-merge-region-keys":0}'

// get config and found config still not changed.
curl http://{PD_IP}/pd/api/v1/config/schedule | grep max-merge-region-keys

  "max-merge-region-keys": 2000,

[3pointer]

cc @rleungx, should PD provide an api to tell us the specified config's ttl worked?

[rleungx]

Yes, we should. See tikv/pd#4596

[sleepymole]

It is the same as #33546 and fixed by #33545.

[3pointer]

it's not as same as scheduler