Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add index block api #58094

Merged
merged 26 commits into from
Jun 30, 2020
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -805,7 +805,8 @@ public void testApiNamingConventions() throws Exception {
"indices.get_data_stream",
"indices.delete_data_stream",
"indices.simulate_template",
"indices.resolve_index"
"indices.resolve_index",
"indices.add_blocks"
};
//These API are not required for high-level client feature completeness
String[] notRequiredApi = new String[] {
Expand Down
31 changes: 2 additions & 29 deletions docs/reference/index-modules.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -173,35 +173,6 @@ specific index module:
for the <<analysis-shingle-tokenfilter,`shingle` token filter>>. Defaults to
`3`.

`index.blocks.read_only`::

Set to `true` to make the index and index metadata read only, `false` to
allow writes and metadata changes.

`index.blocks.read_only_allow_delete`::

Similar to `index.blocks.read_only`, but also allows deleting the index to
make more resources available. The <<disk-based-shard-allocation,disk-based shard
allocator>> may add and remove this block automatically.

Deleting documents from an index to release resources - rather than deleting the index itself - can increase the index size over time. When `index.blocks.read_only_allow_delete` is set to `true`, deleting documents is not permitted. However, deleting the index itself releases the read-only index block and makes resources available almost immediately.

IMPORTANT: {es} adds and removes the read-only index block automatically when the disk utilization falls below the high watermark, controlled by <<cluster-routing-flood_stage,cluster.routing.allocation.disk.watermark.flood_stage>>.

`index.blocks.read`::

Set to `true` to disable read operations against the index.

`index.blocks.write`::

Set to `true` to disable data write operations against the index. Unlike `read_only`,
this setting does not affect metadata. For instance, you can close an index with a `write`
block, but not an index with a `read_only` block.

`index.blocks.metadata`::

Set to `true` to disable index metadata reads and writes.

`index.max_refresh_listeners`::

Maximum number of refresh listeners available on each shard of the index.
Expand Down Expand Up @@ -321,6 +292,8 @@ include::index-modules/analysis.asciidoc[]

include::index-modules/allocation.asciidoc[]

include::index-modules/blocks.asciidoc[]

include::index-modules/mapper.asciidoc[]

include::index-modules/merge.asciidoc[]
Expand Down
135 changes: 135 additions & 0 deletions docs/reference/index-modules/blocks.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
[[index-modules-blocks]]
== Index blocks

Index blocks limit the kind of operations that are available on a certain
index. The blocks come in different flavours, allowing to block write,
read, or metadata operations. The blocks can be set / removed using dynamic
index settings, or can be added using a dedicated API, which also ensures
that, once successfully returning to the user, all shards of the index are
properly accounting for the block, for example that all in-flight writes to
an index have been completed after adding the write block.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call out that this only applies to write operations, not read operations?


[float]
[[index-block-settings]]
=== Index block settings

The following _dynamic_ index settings determine the blocks present on an
index:

`index.blocks.read_only`::

Set to `true` to make the index and index metadata read only, `false` to
allow writes and metadata changes.

`index.blocks.read_only_allow_delete`::

Similar to `index.blocks.read_only`, but also allows deleting the index to
make more resources available. The <<disk-based-shard-allocation,disk-based shard
allocator>> may add and remove this block automatically.

Deleting documents from an index to release resources - rather than deleting the index itself - can increase the index size over time. When `index.blocks.read_only_allow_delete` is set to `true`, deleting documents is not permitted. However, deleting the index itself releases the read-only index block and makes resources available almost immediately.

IMPORTANT: {es} adds and removes the read-only index block automatically when the disk utilization falls below the high watermark, controlled by <<cluster-routing-flood_stage,cluster.routing.allocation.disk.watermark.flood_stage>>.

`index.blocks.read`::

Set to `true` to disable read operations against the index.

`index.blocks.write`::

Set to `true` to disable data write operations against the index. Unlike `read_only`,
this setting does not affect metadata. For instance, you can close an index with a `write`
block, but not an index with a `read_only` block.
jrodewig marked this conversation as resolved.
Show resolved Hide resolved

`index.blocks.metadata`::

Set to `true` to disable index metadata reads and writes.

[float]
[[add-index-block]]
=== Add index block API

Adds an index block, and ensures that, once successfully returning to the
user, all shards of the index are properly accounting for the block, for
example that all in-flight writes to an index have been completed after
adding the write block.
jrodewig marked this conversation as resolved.
Show resolved Hide resolved

[source,console]
--------------------------------------------------
PUT /twitter/_blocks/write
--------------------------------------------------
// TEST[setup:twitter]

The blocks available are `read_only`, `read_only_allow_delete`, `read`, `write`,
and `metadata`, and their meaning is described above in the settings section.
jrodewig marked this conversation as resolved.
Show resolved Hide resolved


[[add-index-block-api-request]]
==== {api-request-title}

`PUT /<index>/_blocks/<block>`


[[add-index-block-api-desc]]
==== {api-description-title}

You use the add index blocks API to add an index block.

include::{es-repo-dir}/indices/open-close.asciidoc[tag=closed-index]


[[add-index-block-api-path-params]]
==== {api-path-parms-title}

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index]
+
To add blocks to all indices, use `_all` or `*`. To disallow the adding
of blocks to indices with `_all` or wildcard expressions,
change the `action.destructive_requires_name` cluster setting to `true`.
You can update this setting in the `elasticsearch.yml` file
or using the <<cluster-update-settings,cluster update settings>> API.

jrodewig marked this conversation as resolved.
Show resolved Hide resolved

[[add-index-block-api-query-params]]
==== {api-query-parms-title}

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
+
Defaults to `true`.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
+
Defaults to `open`.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=timeoutparms]


[[add-index-block-api-example]]
==== {api-examples-title}

The following example shows how to add an index block:

[source,console]
--------------------------------------------------
PUT /my_index/_blocks/write
--------------------------------------------------
// TEST[s/^/PUT my_index\n/]

The API returns following response:

[source,console-result]
--------------------------------------------------
{
"acknowledged" : true,
"shards_acknowledged" : true,
"indices" : {
"my_index" : {
"blocked" : true
}
}
}
--------------------------------------------------


Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
{
"indices.add_blocks":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-blocks.html",
"description":"Adds a block to an index."
},
"stability":"stable",
"url":{
"paths":[
{
"path":"/{index}/_blocks/{block}",
"methods":[
"PUT"
],
"parts":{
"index":{
"type":"list",
"description":"A comma separated list of indices to add blocks to"
},
"block":{
"type":"string",
"description":"The block to add (one of read, write, read_only, metadata, read_only_allow_delete)"
}
}
}
]
},
"params":{
"timeout":{
"type":"time",
"description":"Explicit operation timeout"
},
"master_timeout":{
"type":"time",
"description":"Specify timeout for connection to master"
},
"ignore_unavailable":{
"type":"boolean",
"description":"Whether specified concrete indices should be ignored when unavailable (missing or closed)"
},
"allow_no_indices":{
"type":"boolean",
"description":"Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes `_all` string or when no indices have been specified)"
},
"expand_wildcards":{
"type":"enum",
"options":[
"open",
"closed",
"hidden",
"none",
"all"
],
"default":"open",
"description":"Whether to expand wildcard expression to concrete indices that are open, closed or both."
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
"Basic test for index blocks":
- skip:
version: " - 7.99.99"
reason: "index block APIs have only been made available in 8.0.0"
- do:
indices.create:
index: test_index
body:
settings:
number_of_replicas: 0

- do:
indices.add_blocks:
index: test_index
block: write
- is_true: acknowledged

- do:
catch: /cluster_block_exception/
index:
index: test_index
body: { foo: bar }

- do:
search:
index: test_index

- do:
indices.put_settings:
index: test_index
body:
index.blocks.write: false
Loading