diff --git a/docs/reference/frozen-indices.asciidoc b/docs/reference/frozen-indices.asciidoc deleted file mode 100644 index 3922fc18dff3..000000000000 --- a/docs/reference/frozen-indices.asciidoc +++ /dev/null @@ -1,110 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[frozen-indices]] -= Frozen indices - -[partintro] --- -{es} indices keep some data structures in memory to allow you to search them -efficiently and to index into them. If you have a lot of indices then the -memory required for these data structures can add up to a significant amount. -For indices that are searched frequently it is better to keep these structures -in memory because it takes time to rebuild them. However, you might access some -of your indices so rarely that you would prefer to release the corresponding -memory and rebuild these data structures on each search. - -For example, if you are using time-based indices to store log messages or time -series data then it is likely that older indices are searched much less often -than the more recent ones. Older indices also receive no indexing requests. -Furthermore, it is usually the case that searches of older indices are for -performing longer-term analyses for which a slower response is acceptable. - -If you have such indices then they are good candidates for becoming _frozen -indices_. {es} builds the transient data structures of each shard of a frozen -index each time that shard is searched, and discards these data structures as -soon as the search is complete. Because {es} does not maintain these transient -data structures in memory, frozen indices consume much less heap than normal -indices. This allows for a much higher disk-to-heap ratio than would otherwise -be possible. - -You can freeze the index using the <>. - -Searches performed on frozen indices use the small, dedicated, -<> to control the number of -concurrent searches that hit frozen shards on each node. This limits the amount -of extra memory required for the transient data structures corresponding to -frozen shards, which consequently protects nodes against excessive memory -consumption. - -Frozen indices are read-only: you cannot index into them. - -Searches on frozen indices are expected to execute slowly. Frozen indices are -not intended for high search load. It is possible that a search of a frozen -index may take seconds or minutes to complete, even if the same searches -completed in milliseconds when the indices were not frozen. - -To make a frozen index writable again, use the <>. - --- - -[role="xpack"] -[testenv="basic"] -[[best_practices]] -== Best practices - -Since frozen indices provide a much higher disk to heap ratio at the expense of search latency, it is advisable to allocate frozen indices to -dedicated nodes to prevent searches on frozen indices influencing traffic on low latency nodes. There is significant overhead in loading -data structures on demand which can cause page faults and garbage collections, which further slow down query execution. - -Since indices that are eligible for freezing are unlikely to change in the future, disk space can be optimized as described in <>. - -It's highly recommended to <> your indices prior to freezing to ensure that each shard has only a single -segment on disk. This not only provides much better compression but also simplifies the data structures needed to service aggregation -or sorted search requests. - -[source,console] --------------------------------------------------- -POST /my-index-000001/_forcemerge?max_num_segments=1 --------------------------------------------------- -// TEST[setup:my_index] - -[role="xpack"] -[testenv="basic"] -[[searching_a_frozen_index]] -== Searching a frozen index - -Frozen indices are throttled in order to limit memory consumptions per node. The number of concurrently loaded frozen indices per node is -limited by the number of threads in the <> threadpool, which is `1` by default. -Search requests will not be executed against frozen indices by default, even if a frozen index is named explicitly. This is -to prevent accidental slowdowns by targeting a frozen index by mistake. To include frozen indices a search request must be executed with -the query parameter `ignore_throttled=false`. - -[source,console] --------------------------------------------------- -GET /my-index-000001/_search?q=user.id:kimchy&ignore_throttled=false --------------------------------------------------- -// TEST[setup:my_index] - -[role="xpack"] -[testenv="basic"] -[[monitoring_frozen_indices]] -== Monitoring frozen indices - -Frozen indices are ordinary indices that use search throttling and a memory efficient shard implementation. For API's like the -<> frozen indices may identified by an index's `search.throttled` property (`sth`). - -[source,console] --------------------------------------------------- -GET /_cat/indices/my-index-000001?v=true&h=i,sth --------------------------------------------------- -// TEST[s/^/PUT my-index-000001\nPOST my-index-000001\/_freeze\n/] - -The response looks like: - -[source,txt] --------------------------------------------------- -i sth -my-index-000001 true --------------------------------------------------- -// TESTRESPONSE[non_json] - diff --git a/docs/reference/glossary.asciidoc b/docs/reference/glossary.asciidoc index 8fb972d52a8a..949cfb95a320 100644 --- a/docs/reference/glossary.asciidoc +++ b/docs/reference/glossary.asciidoc @@ -155,21 +155,6 @@ Manually triggers a <> to reduce the number of See the {ref}/indices-forcemerge.html[force merge API]. // end::force-merge-def[] -[[glossary-freeze]] freeze:: -// tag::freeze-def[] -// tag::freeze-def-short[] -Makes an index read-only and minimizes its memory footprint. -// end::freeze-def-short[] -See the {ref}/freeze-index-api.html[freeze API]. -// end::freeze-def[] - -[[glossary-frozen-index]] frozen index:: -// tag::frozen-index-def[] -An <> reduced to a low overhead state that still enables -occasional searches. -See the {ref}/freeze-index-api.html[freeze API]. -// end::frozen-index-def[] - [[glossary-frozen-phase]] frozen phase:: // tag::frozen-phase-def[] Fourth possible phase in the <>. In diff --git a/docs/reference/ilm/actions/ilm-freeze.asciidoc b/docs/reference/ilm/actions/ilm-freeze.asciidoc index abc5b4ce4a6c..51c26ec4e92b 100644 --- a/docs/reference/ilm/actions/ilm-freeze.asciidoc +++ b/docs/reference/ilm/actions/ilm-freeze.asciidoc @@ -4,7 +4,7 @@ Phases allowed: cold. -<> an index to minimize its memory footprint. +<> an index. IMPORTANT: Freezing an index closes the index and reopens it within the same API call. This means that for a short time no primaries are allocated. diff --git a/docs/reference/ilm/ilm-overview.asciidoc b/docs/reference/ilm/ilm-overview.asciidoc index c67db1a90bf6..ced0e05e4868 100644 --- a/docs/reference/ilm/ilm-overview.asciidoc +++ b/docs/reference/ilm/ilm-overview.asciidoc @@ -18,8 +18,7 @@ include::../glossary.asciidoc[tag=rollover-def-short] include::../glossary.asciidoc[tag=shrink-def-short] * **Force merge**: include::../glossary.asciidoc[tag=force-merge-def-short] -* **Freeze**: -include::../glossary.asciidoc[tag=freeze-def-short] +* **Freeze**: <> an index and makes it read-only. * **Delete**: Permanently remove an index, including all of its data and metadata. {ilm-init} makes it easier to manage indices in hot-warm-cold architectures, diff --git a/docs/reference/index.asciidoc b/docs/reference/index.asciidoc index 7efb20fdafa1..ce2055b81be4 100644 --- a/docs/reference/index.asciidoc +++ b/docs/reference/index.asciidoc @@ -53,8 +53,6 @@ include::autoscaling/index.asciidoc[] include::monitoring/index.asciidoc[] -include::frozen-indices.asciidoc[] - include::data-rollup-transform.asciidoc[] include::high-availability.asciidoc[] diff --git a/docs/reference/indices/apis/freeze.asciidoc b/docs/reference/indices/apis/freeze.asciidoc index 6e2c3c9760c2..f70ead245a7c 100644 --- a/docs/reference/indices/apis/freeze.asciidoc +++ b/docs/reference/indices/apis/freeze.asciidoc @@ -6,7 +6,9 @@ Freeze index ++++ +// tag::freeze-api-dep[] deprecated::[7.14, Frozen indices are no longer useful due to https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage[recent improvements in heap memory usage].] +// end::freeze-api-dep[] Freezes an index. @@ -27,7 +29,7 @@ Freezes an index. A frozen index has almost no overhead on the cluster (except for maintaining its metadata in memory) and is read-only. Read-only indices are blocked for write operations, such as <> or <>. See <> and <>. +merges>>. See <>. The current write index on a data stream cannot be frozen. In order to freeze the current write index, the data stream must first be diff --git a/docs/reference/indices/apis/unfreeze.asciidoc b/docs/reference/indices/apis/unfreeze.asciidoc index b4ea421b0955..cc62951f4d60 100644 --- a/docs/reference/indices/apis/unfreeze.asciidoc +++ b/docs/reference/indices/apis/unfreeze.asciidoc @@ -23,7 +23,7 @@ Unfreezes an index. ==== {api-description-title} When a frozen index is unfrozen, the index goes through the normal recovery -process and becomes writeable again. See <> and <>. +process and becomes writeable again. See <>. IMPORTANT: Freezing an index will close the index and reopen it within the same API call. This causes primaries to not be allocated for a short amount of time diff --git a/docs/reference/indices/index-mgmt.asciidoc b/docs/reference/indices/index-mgmt.asciidoc index 0dcfbe045560..1b5cc2638980 100644 --- a/docs/reference/indices/index-mgmt.asciidoc +++ b/docs/reference/indices/index-mgmt.asciidoc @@ -43,7 +43,7 @@ Open {kib}'s main menu and click *Stack Management > Index Management*. image::images/index-mgmt/management_index_labels.png[Index Management UI] The *Index Management* page contains an overview of your indices. -Badges indicate if an index is <>, a +Badges indicate if an index is <>, a <>, or a <>. diff --git a/docs/reference/mapping/params/eager-global-ordinals.asciidoc b/docs/reference/mapping/params/eager-global-ordinals.asciidoc index d1e3f07c4570..61ab357e09c0 100644 --- a/docs/reference/mapping/params/eager-global-ordinals.asciidoc +++ b/docs/reference/mapping/params/eager-global-ordinals.asciidoc @@ -85,7 +85,7 @@ PUT my-index-000001/_mapping ------------ // TEST[continued] -IMPORTANT: On a <>, global ordinals are discarded +IMPORTANT: On a <>, global ordinals are discarded after each search and rebuilt again when they're requested. This means that `eager_global_ordinals` should not be used on frozen indices: it would cause global ordinals to be reloaded on every search. Instead, the index should diff --git a/docs/reference/redirects.asciidoc b/docs/reference/redirects.asciidoc index 2e121b64bcdf..53dba405d01c 100644 --- a/docs/reference/redirects.asciidoc +++ b/docs/reference/redirects.asciidoc @@ -1536,3 +1536,27 @@ See <>. === Settng JVM options See <>. + +[role="exclude",id="frozen-indices"] +=== Frozen indices + +// tag::frozen-index-redirect[] +include::{es-repo-dir}/indices/apis/freeze.asciidoc[tag=freeze-api-dep] + +For API documentation, see <> and <>. +// end::frozen-index-redirect[] + +[role="exclude",id="best_practices"] +=== Best practices for frozen indices + +include::redirects.asciidoc[tag=frozen-index-redirect] + +[role="exclude",id="searching_a_frozen_index"] +=== Searching a frozen index + +include::redirects.asciidoc[tag=frozen-index-redirect] + +[role="exclude",id="monitoring_frozen_indices"] +=== Monitoring frozen indices + +include::redirects.asciidoc[tag=frozen-index-redirect] diff --git a/docs/reference/search/search-your-data/long-running-searches.asciidoc b/docs/reference/search/search-your-data/long-running-searches.asciidoc index 6f7cf7036524..7a2af0389573 100644 --- a/docs/reference/search/search-your-data/long-running-searches.asciidoc +++ b/docs/reference/search/search-your-data/long-running-searches.asciidoc @@ -5,7 +5,7 @@ {es} generally allows you to quickly search across big amounts of data. There are situations where a search executes on many shards, possibly against -<> and spanning multiple +<> and spanning multiple <>, for which results are not expected to be returned in milliseconds. When you need to execute long-running searches, synchronously diff --git a/docs/reference/search/search-your-data/search-your-data.asciidoc b/docs/reference/search/search-your-data/search-your-data.asciidoc index 8717fe8dd9e0..29e90528c627 100644 --- a/docs/reference/search/search-your-data/search-your-data.asciidoc +++ b/docs/reference/search/search-your-data/search-your-data.asciidoc @@ -221,7 +221,7 @@ _synchronous_ by default. The search request waits for complete results before returning a response. However, complete results can take longer for searches across -<> or <> or <>. To avoid long waits, you can run an _asynchronous_, or _async_, search diff --git a/docs/reference/sql/endpoints/jdbc.asciidoc b/docs/reference/sql/endpoints/jdbc.asciidoc index 15c6f950deff..7b0e74cd4951 100644 --- a/docs/reference/sql/endpoints/jdbc.asciidoc +++ b/docs/reference/sql/endpoints/jdbc.asciidoc @@ -145,7 +145,7 @@ will be - typically the first in natural ascending order) for fields with multip [discrete] ==== Index -`index.include.frozen` (default `false`):: Whether to include <> in the query execution or not (default). +`index.include.frozen` (default `false`):: Whether to include <> in the query execution or not (default). [discrete] ==== Additional diff --git a/docs/reference/sql/endpoints/rest.asciidoc b/docs/reference/sql/endpoints/rest.asciidoc index 77d9be3d2905..7ca7e70774b7 100644 --- a/docs/reference/sql/endpoints/rest.asciidoc +++ b/docs/reference/sql/endpoints/rest.asciidoc @@ -612,7 +612,7 @@ More information available https://docs.oracle.com/javase/8/docs/api/java/time/Z |index_include_frozen |false -|Whether to include <> in the query execution or not (default). +|Whether to include <> in the query execution or not (default). |params |none diff --git a/docs/reference/sql/language/indices.asciidoc b/docs/reference/sql/language/indices.asciidoc index e8d20260d0ac..c275f7ed92bd 100644 --- a/docs/reference/sql/language/indices.asciidoc +++ b/docs/reference/sql/language/indices.asciidoc @@ -88,12 +88,8 @@ requires the keyword `LIKE` for SQL `LIKE` pattern. [[sql-index-frozen]] === Frozen Indices -{es} <> are a useful and powerful tool for hot/warm architecture introduced in {es} 6.6, -essentially by trading speed for memory. -{es-sql} supports frozen indices and similar to {es}, due to their performance characteristics, allows searches on them only -when explicitly told so by user - in other words, by default, frozen indices are not included in searches. - -One can toggle the use of frozen indices through: +By default, {es-sql} doesn't search <>. To +search frozen indices, use one of the following features: dedicated configuration parameter:: Set to `true` properties `index_include_frozen` in the <> or `index.include.frozen` in the drivers to include frozen indices.