Shrink an index from a snapshot #73500
Labels
:Distributed/Recovery
Anything around constructing a new shard, either from a local or a remote source.
>enhancement
Team:Distributed
Meta label for distributed team
In order to help address DTS costs, and also reduce the complexity of ILM's shrink operation, we should investigate adding the ability to shrink an index from a snapshot rather than from local disk.
It would be nice if instead of shards having to be colocated, we could take a snapshot of the index, and then perform a restore and shrink at the same time. This would not only alleviate #63519 but it would also be much less error prone to issues like a situation where not all the shards can fit on a node.
In order to accomplish this, we would need to introduce the Default Repository (#66040), so that a repository does not need to be specified for the shrink request.
Note that this diagram shows the combination of shrink and forcemerge as described in #73499, but this is not a prerequisite, as the two are independent.
One extremely beneficial side effect of implementing shrink like this is that shrink would no longer need to perform the slightly error-prone behavior of identifying a single node to move shards prior to the shrink (which itself can cost users money depending on where their data resides). It also would remove some code complexity on the ES side for handling shrink behavior during cluster/node shutdowns.
The text was updated successfully, but these errors were encountered: