Cluster doesn't accept new client nodes or update settings API calls while initializing or rebalancing shards.

[Awnedion]

When an ElasticSearch cluster is initializing shards or relocating them, it will not let new client nodes join the cluster or any update settings API calls take effect until all initializing and rebalancing is complete. When this process starts to take hours due to a large amount of indices/shards to initialize on startup, this problem creates an unnecessarily long amount of downtime because even once the cluster reaches yellow or even green status, no client nodes can connect.

[imotov]

Which version of Elasticsearch are you using? Did you change anything in elasticsearch configuration?

[Awnedion]

I'm using version 0.20.5.

cluster_concurrent_rebalance: 20
node_concurrent_recoveries: 150
node_initial_primaries_recoveries:150

indices.recovery.concurrent_streams: 10

Unicast discovery:
ping_interval: 5s
ping_timeout: 60s
ping_retries: 5

blocking thread pool
auto_import_dangled: no

[folke]

Just wanted to say we're having the same issue. This is actually a big issue for us since we have a lot of shards (20k in total). We just added 10 new servers and it takes forever to rebalance them. Bulk operations and other thing simply timeout after a while. We're on 0.20.6

[clintongormley]

Trying to do too much I/O at once is always going to hurt responsiveness. I/O needs to be throttled to suit your hardware. This will also be helped by the addition of sequence numbers to speed up recovery. Closing in favour of #6069