Skip to content

Commit

Permalink
Reduce shard inactivity timeout to 5m
Browse files Browse the repository at this point in the history
To better distribute the memory allocating to indexing, the IndexingMemoryController periodically checks the different shard for their last indexing activity. If no activity has happened for a while, the controller marks the shards as in active and allocated it's memory buffer budget (but a small minimal budget)  to other active shards.  The recently added synced flush feature (#11179, #11336) uses this inactivity trigger to attempt as  a trigger to attempt adding a sync id marker (which will speed up future recoveries).

We wait for 30m before declaring a shard inactive. However, these days the operation just requires a refresh and is light. We can be stricter (and 5m) increase the chance a synced flush will be triggered.

Closes #11479
  • Loading branch information
bleskes committed Jun 3, 2015
1 parent 01f20e0 commit 06949d6
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
4 changes: 2 additions & 2 deletions docs/reference/indices/flush.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ POST /_flush
=== Synced Flush

Elasticsearch tracks the indexing activity of each shard. Shards that have not
received any indexing operations for 30 minutes are automatically marked as inactive. This presents
received any indexing operations for 5 minutes are automatically marked as inactive. This presents
an opportunity for Elasticsearch to reduce shard resources and also perform
a special kind of flush, called `synced flush`. A synced flush performs a normal flush, then adds
a generated unique marker (sync_id) to all shards.
Expand Down Expand Up @@ -117,7 +117,7 @@ which returns something similar to:
=== Synced Flush API

The Synced Flush API allows an administrator to initiate a synced flush manually. This can be particularly useful for
a planned (rolling) cluster restart where you can stop indexing and don't want to wait the default 30 minutes for
a planned (rolling) cluster restart where you can stop indexing and don't want to wait the default 5 minutes for
idle indices to be sync-flushed automatically.

While handy, there are a couple of caveats for this API:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ public IndexingMemoryController(Settings settings, ThreadPool threadPool, Indice
this.minShardTranslogBufferSize = componentSettings.getAsBytesSize("min_shard_translog_buffer_size", new ByteSizeValue(2, ByteSizeUnit.KB));
this.maxShardTranslogBufferSize = componentSettings.getAsBytesSize("max_shard_translog_buffer_size", new ByteSizeValue(64, ByteSizeUnit.KB));

this.inactiveTime = componentSettings.getAsTime("shard_inactive_time", TimeValue.timeValueMinutes(30));
this.inactiveTime = componentSettings.getAsTime("shard_inactive_time", TimeValue.timeValueMinutes(5));
// we need to have this relatively small to move a shard from inactive to active fast (enough)
this.interval = componentSettings.getAsTime("interval", TimeValue.timeValueSeconds(30));

Expand Down

0 comments on commit 06949d6

Please sign in to comment.