Upgrade Assistant cannot reindex Elasticsearch's .tasks index

[joshdover]

Kibana version: 6.7.0

Related to #26368

Reindexing the .tasks index in Elasticsearch is currently not possible through the Upgrade Assistant UI.

For the reindexing feature in the 7.0 Upgrade Assistant, we utilize the Reindex API's wait_for_completion=false URL param in order to create an ES Task for the reindex. However, this does not work when reindexing the .tasks index itself, because before we begin reindexing, the source index is set to index.blocks.writes: true.

Before 6.7.0 FF, we will need to solve this problem. Two possible solutions:

• When upgrading the .tasks index only, do not use wait_for_completion=false and instead wait for the Reindex API to return. This "works" but it's possible for this request to timeout, and then in that case it's impossible to know for sure that the reindex finished. The best we can do is compare counts between the source and destination indices. There is also a risk that blocking writes to the .tasks index will break other stack features. An example of this approach is here.
• Add support for this in an Elasticsearch API, preferably the Migration Upgrade API. This would probably need to handle "live reindexes" for this index somehow. I only see this being necessary if blocking writes to the .tasks index is not an option at all. Note, that there is some necessary work that would need to be done in Kibana to support this API.

[elasticmachine]

Pinging @elastic/kibana-operations