diff --git a/docs/reference/migration/index.asciidoc b/docs/reference/migration/index.asciidoc index c4bb1a5c21b7b..33d3c9553dc70 100644 --- a/docs/reference/migration/index.asciidoc +++ b/docs/reference/migration/index.asciidoc @@ -1,8 +1,9 @@ [[breaking-changes]] = Migration guide -[partintro] --- +:ess-setting-change: {ess-icon} indicates a change to a supported {cloud}/ec-add-user-settings.html[user setting] for {ess}. +:ess-skip-section: If you use {ess}, skip this section. {ess} handles these changes for you. + This section discusses the changes that you need to be aware of to migrate your application to {version}. For more information about what's new in this release, see the <> and <>. @@ -38,6 +39,5 @@ For information about how to upgrade your cluster, see <>. * <> * <> --- include::migrate_8_1.asciidoc[] include::migrate_8_0.asciidoc[] diff --git a/docs/reference/migration/migrate_8_0.asciidoc b/docs/reference/migration/migrate_8_0.asciidoc index 67e6df83e7eb3..13f14477afd67 100644 --- a/docs/reference/migration/migrate_8_0.asciidoc +++ b/docs/reference/migration/migrate_8_0.asciidoc @@ -9,38 +9,7 @@ your application to {es} 8.0. See also <> and <>. -coming[8.0.0] - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> +coming::[8.0.0] [discrete] [[breaking-changes-8.0]] @@ -58,93 +27,18 @@ the old behavior is supported until the next major release. To find out if you are using any deprecated functionality, enable <>. -//NOTE: The notable-breaking-changes tagged regions are re-used in the -//Installation and Upgrade Guide - -//tag::notable-breaking-changes[] - -.Indices created in {es} 6.x and earlier versions are not supported. -[%collapsible] -==== -*Details* + -Elasticsearch 8.0 can read indices created in version 7.0 or above. An -Elasticsearch 8.0 node will not start in the presence of indices created in a -version of Elasticsearch before 7.0. - -*Impact* + -Reindex indices created in {es} 6.x or before with {es} 7.x if they need to be carried forward to {es} 8.x. -==== - -.REST API endpoints containing `_xpack` have been removed. -[%collapsible] -==== -*Details* + -In 7.0, we deprecated REST endpoints that contain `_xpack` in their path. These -endpoints are now removed in 8.0. Each endpoint that was deprecated and removed -is replaced with a new endpoint that does not contain `_xpack`. As an example, -`/{index}/_xpack/graph/_explore` is replaced by `/{index}/_graph/explore`. - -*Impact* + -Use the replacement REST API endpoints. Requests submitted to the `_xpack` -API endpoints will return an error. -==== - -.Several EOL operating systems are no longer supported. -[%collapsible] -==== -*Details* + -The following operating systems have reached their end of life and are no longer -supported by {es}: - -* Amazon Linux -* CentOS 6 -* Debian 8 -* openSUSE Leap 42 -* Oracle Enterprise Linux 6 -* Ubuntu 16.04 - -We've also removed support for `SysV init`. No supported operating systems use -the `SysV init` process. - -*Details* + -Ensure your nodes use a -https://www.elastic.co/support/matrix#matrix_os[supported operating system]. -Running {es} on an unsupported operating system can result in unexpected errors -or failures. -==== - -// end::notable-breaking-changes[] - -include::migrate_8_0/aggregations.asciidoc[] -include::migrate_8_0/allocation.asciidoc[] -include::migrate_8_0/analysis.asciidoc[] -include::migrate_8_0/breaker.asciidoc[] -include::migrate_8_0/cluster.asciidoc[] -include::migrate_8_0/ccr.asciidoc[] -include::migrate_8_0/discovery.asciidoc[] -include::migrate_8_0/eql.asciidoc[] -include::migrate_8_0/http.asciidoc[] -include::migrate_8_0/ilm.asciidoc[] -include::migrate_8_0/indices.asciidoc[] -include::migrate_8_0/ingest.asciidoc[] -include::migrate_8_0/java.asciidoc[] -include::migrate_8_0/logging.asciidoc[] -include::migrate_8_0/mappings.asciidoc[] -include::migrate_8_0/network.asciidoc[] -include::migrate_8_0/node.asciidoc[] -include::migrate_8_0/packaging.asciidoc[] -include::migrate_8_0/reindex.asciidoc[] -include::migrate_8_0/api.asciidoc[] -include::migrate_8_0/rollup.asciidoc[] -include::migrate_8_0/scripting.asciidoc[] -include::migrate_8_0/search.asciidoc[] -include::migrate_8_0/security.asciidoc[] -include::migrate_8_0/settings.asciidoc[] -include::migrate_8_0/snapshots.asciidoc[] -include::migrate_8_0/threadpool.asciidoc[] +include::migrate_8_0/_cluster-node-setting-changes.asciidoc[] +include::migrate_8_0/_command-line-tool-changes.asciidoc[] +include::migrate_8_0/_index-setting-changes.asciidoc[] +include::migrate_8_0/_java-api-changes.asciidoc[] +include::migrate_8_0/_jvm-option-changes.asciidoc[] +include::migrate_8_0/_logging-changes.asciidoc[] +include::migrate_8_0/_mapping-changes.asciidoc[] +include::migrate_8_0/_packaging-changes.asciidoc[] +include::migrate_8_0/_painless-changes.asciidoc[] +include::migrate_8_0/_rest-api-changes.asciidoc[] +include::migrate_8_0/_system-req-changes.asciidoc[] include::migrate_8_0/transform.asciidoc[] -include::migrate_8_0/transport.asciidoc[] -include::migrate_8_0/watcher.asciidoc[] [discrete] [[deprecated-8.0]] @@ -161,9 +55,12 @@ the old behavior is supported until the next major release. To find out if you are using any deprecated functionality, enable <>. +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide +//tag::notable-breaking-changes[] [discrete] -[[breaking_80_settings_deprecations]] -==== Settings deprecations +[[breaking_80_cluster_node_setting_deprecations]] +==== Cluster and node setting deprecations [[deprecate-transient-cluster-settings]] .Transient cluster settings are deprecated. @@ -183,6 +80,31 @@ settings instead. See the {ref}/transient-settings-migration-guide.html[Transient settings migration guide]. ==== +//end::notable-breaking-changes[] + +[discrete] +[[breaking_80_command_line_tool_deprecations]] +==== Command line tool deprecations + +TIP: {ess-skip-section} + +[[deprecate-elasticsearch-setup-passwords]] +.The `elasticsearch-setup-passwords` tool is deprecated. +[%collapsible] +==== +*Details* + +The `elasticsearch-setup-passwords` tool is deprecated in 8.0. To +manually reset the password for built-in users (including the `elastic` user), use +the {ref}/reset-password.html[`elasticsearch-reset-password`] tool, the {es} +{ref}/security-api-change-password.html[change passwords API], or the +User Management features in {kib}. +`elasticsearch-setup-passwords` will be removed in a future release. + +*Impact* + +Passwords are generated automatically for the `elastic` user when you start {es} for the first time. If you run `elasticsearch-setup-passwords` after +starting {es}, it will fail because the `elastic` +user password is already configured. +==== include::migrate_8_0/migrate_to_java_time.asciidoc[] include::transient-settings-migration-guide.asciidoc[] diff --git a/docs/reference/migration/migrate_8_0/_cluster-node-setting-changes.asciidoc b/docs/reference/migration/migrate_8_0/_cluster-node-setting-changes.asciidoc new file mode 100644 index 0000000000000..f5b7bf89a594d --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_cluster-node-setting-changes.asciidoc @@ -0,0 +1,880 @@ +[discrete] +[[breaking_80_cluster_node_setting_changes]] +==== Cluster and node setting changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +TIP: {ess-setting-change} + +.You can no longer set `xpack.searchable.snapshot.shared_cache.size` on non-frozen nodes. {ess-icon} +[%collapsible] +==== +*Details* + +Setting `xpack.searchable.snapshot.shared_cache.size` on a node +that does not have the `data_frozen` role was deprecated in {es} 7.12.0 and has +been removed in {es} 8.0.0. + +*Impact* + +{es} only allocates partially mounted indices to nodes with the `data_frozen` +role. Remove the `xpack.searchable.snapshot.shared_cache.size` setting from nodes that don't have the `data_frozen` role. +==== + +.`action.destructive_requires_name` now defaults to `false`. {ess-icon} +[%collapsible] +==== +*Details* + +The default for the `action.destructive_requires_name` setting changes from `false` +to `true` in {es} 8.0.0. + +Previously, defaulting to `false` allowed users to use wildcard +patterns to delete, close, or change index blocks on indices. +To prevent the accidental deletion of indices that happen to match a +wildcard pattern, we now default to requiring that destructive +operations explicitly name the indices to be modified. + +*Impact* + +To use wildcard patterns for destructive actions, set +`action.destructive_requires_name` to `false` using the +{ref}/cluster-update-settings.html[] cluster settings API]. +==== + +[[max_clause_count_change]] +.The `indices.query.bool.max_clause_count` setting now limits all query clauses. +[%collapsible] +==== +*Details* + +Previously, the `indices.query.bool.max_clause_count` would apply to the number +of clauses of a single `bool` query. It now applies to the total number of +clauses of the rewritten query. To reduce chances of breaks, its +default value has been bumped from 1024 to 4096. + +*Impact* + +Queries with many clauses should be avoided whenever possible. +If you previously bumped this setting to accommodate heavy queries, +you might need to increase it further. +==== + +[[ilm-poll-interval-limit]] +.`indices.lifecycle.poll_interval` must be greater than `1s`. +[%collapsible] +==== +*Details* + +Setting `indices.lifecycle.poll_interval` too low can cause +excessive load on a cluster. The poll interval must now be at least `1s` (one second). + +*Impact* + +Set `indices.lifecycle.poll_interval` setting to `1s` or +greater in `elasticsearch.yml` or through the +{ref}/cluster-update-settings.html[cluster update settings API]. + +Setting `indices.lifecycle.poll_interval` to less than `1s` in +`elasticsearch.yml` will result in an error on startup. +{ref}/cluster-update-settings.html[Cluster update settings API] requests that +set `indices.lifecycle.poll_interval` to less than `1s` will return an error. +==== + +.The file and native realms are now enabled unless explicitly disabled. +[%collapsible] +==== +*Details* + +The file and native realms are now enabled unless explicitly disabled. If +explicitly disabled, the file and native realms remain disabled at all times. + +Previously, the file and native realms had the following implicit behaviors: + +* If the file and native realms were not configured, they were implicitly disabled +if any other realm was configured. + +* If no other realm was available because realms were either not configured, +not permitted by license, or explicitly disabled, the file and native realms +were enabled, even if explicitly disabled. + +*Impact* + +To explicitly disable the file or native realm, set the respective +`file..enabled` or `native..enabled` setting to `false` +under the `xpack.security.authc.realms` namespace in `elasticsearch.yml`. + +The following configuration example disables the native realm and the file realm. + +[source,yaml] +---- +xpack.security.authc.realms: + + native.realm1.enabled: false + file.realm2.enabled: false + + ... +---- +==== + +.The realm `order` setting is now required. +[%collapsible] +==== +*Details* + +The `xpack.security.authc.realms.{type}.{name}.order` setting is now required and must be +specified for each explicitly configured realm. Each value must be unique. + +*Impact* + +The cluster will fail to start if the requirements are not met. + +For example, the following configuration is invalid: +[source,yaml] +-------------------------------------------------- +xpack.security.authc.realms.kerberos.kerb1: + keytab.path: es.keytab + remove_realm_name: false +-------------------------------------------------- + +And must be configured as: +[source,yaml] +-------------------------------------------------- +xpack.security.authc.realms.kerberos.kerb1: + order: 0 + keytab.path: es.keytab + remove_realm_name: false +-------------------------------------------------- +==== + +[[breaking_80_allocation_change_include_relocations_removed]] +.`cluster.routing.allocation.disk.include_relocations` has been removed. +[%collapsible] +==== +*Details* + +{es} now always accounts for the sizes of relocating shards when making +allocation decisions based on the disk usage of the nodes in the cluster. In +earlier versions, you could disable this by setting `cluster.routing.allocation.disk.include_relocations` to `false`. +That could result in poor allocation decisions that could overshoot watermarks and require significant +extra work to correct. The `cluster.routing.allocation.disk.include_relocations` setting has been removed. + +*Impact* + +Remove the `cluster.routing.allocation.disk.include_relocations` +setting. Specifying this setting in `elasticsearch.yml` will result in an error +on startup. +==== + +.cluster.join.timeout` has been removed. +[%collapsible] +==== +*Details* + +The `cluster.join.timeout` setting has been removed. Join attempts no longer +time out. + +*Impact* + +Remove `cluster.join.timeout` from `elasticsearch.yml`. +==== + +.`discovery.zen` settings have been removed. +[%collapsible] +==== +*Details* + +All settings under the `discovery.zen` namespace are no longer supported. They existed only only for BWC reasons in 7.x. This includes: + +- `discovery.zen.minimum_master_nodes` +- `discovery.zen.no_master_block` +- `discovery.zen.hosts_provider` +- `discovery.zen.publish_timeout` +- `discovery.zen.commit_timeout` +- `discovery.zen.publish_diff.enable` +- `discovery.zen.ping.unicast.concurrent_connects` +- `discovery.zen.ping.unicast.hosts.resolve_timeout` +- `discovery.zen.ping.unicast.hosts` +- `discovery.zen.ping_timeout` +- `discovery.zen.unsafe_rolling_upgrades_enabled` +- `discovery.zen.fd.connect_on_network_disconnect` +- `discovery.zen.fd.ping_interval` +- `discovery.zen.fd.ping_timeout` +- `discovery.zen.fd.ping_retries` +- `discovery.zen.fd.register_connection_listener` +- `discovery.zen.join_retry_attempts` +- `discovery.zen.join_retry_delay` +- `discovery.zen.join_timeout` +- `discovery.zen.max_pings_from_another_master` +- `discovery.zen.send_leave_request` +- `discovery.zen.master_election.wait_for_joins_timeout` +- `discovery.zen.master_election.ignore_non_master_pings` +- `discovery.zen.publish.max_pending_cluster_states` + +*Impact* + +Remove the `discovery.zen` settings from `elasticsearch.yml`. Specifying these settings will result in an error on startup. +==== + +.`http.content_type.required` has been removed. +[%collapsible] +==== +*Details* + +The `http.content_type.required` setting was deprecated in Elasticsearch 6.0 +and has been removed in Elasticsearch 8.0. The setting was introduced in +Elasticsearch 5.3 to prepare users for Elasticsearch 6.0, where content type +auto detection was removed for HTTP requests. + +*Impact* + +Remove the `http.content_type.required` setting from `elasticsearch.yml`. Specifying this setting will result in an error on startup. +==== + +.`http.tcp_no_delay` has been removed. +[%collapsible] +==== +*Details* + +The `http.tcp_no_delay` setting was deprecated in 7.x and has been removed in 8.0. Use`http.tcp.no_delay` instead. + +*Impact* + +Replace the `http.tcp_no_delay` setting with `http.tcp.no_delay`. +Specifying `http.tcp_no_delay` in `elasticsearch.yml` will +result in an error on startup. +==== + +.`network.tcp.connect_timeout` has been removed. +[%collapsible] +==== +*Details* + +The `network.tcp.connect_timeout` setting was deprecated in 7.x and has been removed in 8.0. This setting +was a fallback setting for `transport.connect_timeout`. + +*Impact* + +Remove the`network.tcp.connect_timeout` setting. +Use the `transport.connect_timeout` setting to change the default connection +timeout for client connections. Specifying +`network.tcp.connect_timeout` in `elasticsearch.yml` will result in an +error on startup. +==== + +.`node.max_local_storage_nodes` has been removed. +[%collapsible] +==== +*Details* + +The `node.max_local_storage_nodes` setting was deprecated in 7.x and +has been removed in 8.0. Nodes should be run on separate data paths +to ensure that each node is consistently assigned to the same data path. + +*Impact* + +Remove the `node.max_local_storage_nodes` setting. Specifying this +setting in `elasticsearch.yml` will result in an error on startup. +==== + +[[accept-default-password-removed]] +.The `accept_default_password` setting has been removed. +[%collapsible] +==== +*Details* + +The `xpack.security.authc.accept_default_password` setting has not had any affect +since the 6.0 release of {es} and is no longer allowed. + +*Impact* + +Remove the `xpack.security.authc.accept_default_password` setting from `elasticsearch.yml`. +Specifying this setting will result in an error on startup. +==== + +[[roles-index-cache-removed]] +.The `roles.index.cache.*` settings have been removed. +[%collapsible] +==== +*Details* + +The `xpack.security.authz.store.roles.index.cache.max_size` and +`xpack.security.authz.store.roles.index.cache.ttl` settings have +been removed. These settings have been redundant and deprecated +since the 5.2 release of {es}. + +*Impact* + +Remove the `xpack.security.authz.store.roles.index.cache.max_size` +and `xpack.security.authz.store.roles.index.cache.ttl` settings from `elasticsearch.yml` . +Specifying these settings will result in an error on startup. +==== + +[[separating-node-and-client-traffic]] +.The `transport.profiles.*.xpack.security.type` setting has been removed. +[%collapsible] +==== +*Details* + +The `transport.profiles.*.xpack.security.type` setting is no longer supported. +The Transport Client has been removed and all client traffic now uses +the HTTP transport. Transport profiles using this setting should be removed. + +*Impact* + +Remove the `transport.profiles.*.xpack.security.type` setting from `elasticsearch.yml`. +Specifying this setting in a transport profile will result in an error on startup. +==== + +[discrete] +[[saml-realm-nameid-changes]] +.The `nameid_format` SAML realm setting no longer has a default value. +[%collapsible] +==== +*Details* + +In SAML, Identity Providers (IdPs) can either be explicitly configured to +release a `NameID` with a specific format, or configured to attempt to conform +with the requirements of a Service Provider (SP). The SP declares its +requirements in the `NameIDPolicy` element of a SAML Authentication Request. +In {es}, the `nameid_format` SAML realm setting controls the `NameIDPolicy` +value. + +Previously, the default value for `nameid_format` was +`urn:oasis:names:tc:SAML:2.0:nameid-format:transient`. This setting created +authentication requests that required the IdP to release `NameID` with a +`transient` format. + +The default value has been removed, which means that {es} will create SAML Authentication Requests by default that don't put this requirement on the +IdP. If you want to retain the previous behavior, set `nameid_format` to +`urn:oasis:names:tc:SAML:2.0:nameid-format:transient`. + +*Impact* + +If you currently don't configure `nameid_format` explicitly, it's possible +that your IdP will reject authentication requests from {es} because the requests +do not specify a `NameID` format (and your IdP is configured to expect one). +This mismatch can result in a broken SAML configuration. If you're unsure whether +your IdP is explicitly configured to use a certain `NameID` format and you want to retain current behavior +, try setting `nameid_format` to `urn:oasis:names:tc:SAML:2.0:nameid-format:transient` explicitly. +==== + +.The `xpack.security.transport.ssl.enabled` setting is now required to configure `xpack.security.transport.ssl` settings. +[%collapsible] +==== +*Details* + +It is now an error to configure any SSL settings for +`xpack.security.transport.ssl` without also configuring +`xpack.security.transport.ssl.enabled`. + +*Impact* + +If using other `xpack.security.transport.ssl` settings, you must explicitly +specify the `xpack.security.transport.ssl.enabled` setting. + +If you do not want to enable SSL and are currently using other +`xpack.security.transport.ssl` settings, do one of the following: + +* Explicitly specify `xpack.security.transport.ssl.enabled` as `false` +* Discontinue use of other `xpack.security.transport.ssl` settings + +If you want to enable SSL, follow the instructions in +{ref}/configuring-tls.html#tls-transport[Encrypting communications between nodes +in a cluster]. As part of this configuration, explicitly specify +`xpack.security.transport.ssl.enabled` as `true`. + +For example, the following configuration is invalid: +[source,yaml] +-------------------------------------------------- +xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 +xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 +-------------------------------------------------- + +And must be configured as: +[source,yaml] +-------------------------------------------------- +xpack.security.transport.ssl.enabled: true <1> +xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 +xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 +-------------------------------------------------- +<1> or `false`. +==== + +.The `xpack.security.http.ssl.enabled` setting is now required to configure `xpack.security.http.ssl` settings. +[%collapsible] +==== +*Details* + +It is now an error to configure any SSL settings for +`xpack.security.http.ssl` without also configuring +`xpack.security.http.ssl.enabled`. + +*Impact* + +If using other `xpack.security.http.ssl` settings, you must explicitly +specify the `xpack.security.http.ssl.enabled` setting. + +If you do not want to enable SSL and are currently using other +`xpack.security.http.ssl` settings, do one of the following: + +* Explicitly specify `xpack.security.http.ssl.enabled` as `false` +* Discontinue use of other `xpack.security.http.ssl` settings + +If you want to enable SSL, follow the instructions in +{ref}/configuring-tls.html#tls-http[Encrypting HTTP client communications]. As part +of this configuration, explicitly specify `xpack.security.http.ssl.enabled` +as `true`. + +For example, the following configuration is invalid: +[source,yaml] +-------------------------------------------------- +xpack.security.http.ssl.certificate: elasticsearch.crt +xpack.security.http.ssl.key: elasticsearch.key +xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ] +-------------------------------------------------- + +And must be configured as either: +[source,yaml] +-------------------------------------------------- +xpack.security.http.ssl.enabled: true <1> +xpack.security.http.ssl.certificate: elasticsearch.crt +xpack.security.http.ssl.key: elasticsearch.key +xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ] +-------------------------------------------------- +<1> or `false`. +==== + +.A `xpack.security.transport.ssl` certificate and key are now required to enable SSL for the transport interface. +[%collapsible] +==== +*Details* + +It is now an error to enable SSL for the transport interface without also configuring +a certificate and key through use of the `xpack.security.transport.ssl.keystore.path` +setting or the `xpack.security.transport.ssl.certificate` and +`xpack.security.transport.ssl.key` settings. + +*Impact* + +If `xpack.security.transport.ssl.enabled` is set to `true`, provide a +certificate and key using the `xpack.security.transport.ssl.keystore.path` +setting or the `xpack.security.transport.ssl.certificate` and +`xpack.security.transport.ssl.key` settings. If a certificate and key is not +provided, {es} will return in an error on startup. +==== + +.A `xpack.security.http.ssl` certificate and key are now required to enable SSL for the HTTP server. +[%collapsible] +==== +*Details* + +It is now an error to enable SSL for the HTTP (Rest) server without also configuring +a certificate and key through use of the `xpack.security.http.ssl.keystore.path` +setting or the `xpack.security.http.ssl.certificate` and +`xpack.security.http.ssl.key` settings. + +*Impact* + +If `xpack.security.http.ssl.enabled` is set to `true`, provide a certificate and +key using the `xpack.security.http.ssl.keystore.path` setting or the +`xpack.security.http.ssl.certificate` and `xpack.security.http.ssl.key` +settings. If certificate and key is not provided, {es} will return in an error +on startup. +==== + +.PKCS#11 keystores and trustores cannot be configured in `elasticsearch.yml` +[%collapsible] +==== +*Details* + +The settings `*.ssl.keystore.type` and `*.ssl.truststore.type` no longer accept "PKCS11" as a valid type. +This applies to all SSL settings in Elasticsearch, including + +- `xpack.security.http.keystore.type` +- `xpack.security.transport.keystore.type` +- `xpack.security.http.truststore.type` +- `xpack.security.transport.truststore.type` + +As well as SSL settings for security realms, watcher and monitoring. + +Use of a PKCS#11 keystore or truststore as the JRE's default store is not affected. + +*Impact* + +If you have a PKCS#11 keystore configured within your `elasticsearch.yml` file, you must remove that +configuration and switch to a supported keystore type, or configure your PKCS#11 keystore as the +JRE default store. +==== + +.The `kibana` user has been replaced by `kibana_system`. +[%collapsible] +==== +*Details* + +The `kibana` user was historically used to authenticate {kib} to {es}. +The name of this user was confusing, and was often mistakenly used to login to {kib}. +This has been renamed to `kibana_system` in order to reduce confusion, and to better +align with other built-in system accounts. + +*Impact* + +Replace any use of the `kibana` user with the `kibana_system` user. Specifying +the `kibana` user in `kibana.yml` will result in an error on startup. + +If your `kibana.yml` used to contain: +[source,yaml] +-------------------------------------------------- +elasticsearch.username: kibana +-------------------------------------------------- + +then you should update to use the new `kibana_system` user instead: +[source,yaml] +-------------------------------------------------- +elasticsearch.username: kibana_system +-------------------------------------------------- + +IMPORTANT: The new `kibana_system` user does not preserve the previous `kibana` +user password. You must explicitly set a password for the `kibana_system` user. +==== + +[[search-remote-settings-removed]] +.The `search.remote.*` settings have been removed. +[%collapsible] +==== +*Details* + +In 6.5 these settings were deprecated in favor of `cluster.remote`. In 7.x we +provided automatic upgrading of these settings to their `cluster.remote` +counterparts. In 8.0.0, these settings have been removed. Elasticsearch will +refuse to start if you have these settings in your configuration or cluster +state. + +*Impact* + +Use the replacement `cluster.remote` settings. Discontinue use of the +`search.remote.*` settings. Specifying these settings in `elasticsearch.yml` +will result in an error on startup. +==== + +[[remove-pidfile]] +.The `pidfile` setting has been replaced by `node.pidfile`. +[%collapsible] +==== +*Details* + +To ensure that all settings are in a proper namespace, the `pidfile` setting was +previously deprecated in version 7.4.0 of Elasticsearch, and is removed in +version 8.0.0. Instead, use `node.pidfile`. + +*Impact* + +Use the `node.pidfile` setting. Discontinue use of the `pidfile` setting. +Specifying the `pidfile` setting in `elasticsearch.yml` will result in an error +on startup. +==== + +[[remove-processors]] +.The `processors` setting has been replaced by `node.processors`. +[%collapsible] +==== +*Details* + +To ensure that all settings are in a proper namespace, the `processors` setting +was previously deprecated in version 7.4.0 of Elasticsearch, and is removed in +version 8.0.0. Instead, use `node.processors`. + +*Impact* + +Use the `node.processors` setting. Discontinue use of the `processors` setting. +Specifying the `processors` setting in `elasticsearch.yml` will result in an +error on startup. +==== + +.The `node.processors` setting can no longer exceed the available number of processors. +[%collapsible] +==== +*Details* + +Previously it was possible to set the number of processors used to set the +default sizes for the thread pools to be more than the number of available +processors. As this leads to more context switches and more threads but without +an increase in the number of physical CPUs on which to schedule these additional +threads, the `node.processors` setting is now bounded by the number of available +processors. + +*Impact* + +If specified, ensure the value of `node.processors` setting does not exceed the +number of available processors. Setting the `node.processors` value greater than +the number of available processors in `elasticsearch.yml` will result in an +error on startup. +==== + +.The `cluster.remote.connect` setting has been removed. +[%collapsible] +==== +*Details* + +In Elasticsearch 7.7.0, the setting `cluster.remote.connect` was deprecated in +favor of setting `node.remote_cluster_client`. In Elasticsearch 8.0.0, the +setting `cluster.remote.connect` is removed. + +*Impact* + +Use the `node.remote_cluster_client` setting. Discontinue use of the +`cluster.remote.connect` setting. Specifying the `cluster.remote.connect` +setting in `elasticsearch.yml` will result in an error on startup. +==== + +.The `node.local_storage` setting has been removed. +[%collapsible] +==== +*Details* + +In Elasticsearch 7.8.0, the setting `node.local_storage` was deprecated and +beginning in Elasticsearch 8.0.0 all nodes will require local storage. Therefore, +the `node.local_storage` setting has been removed. + +*Impact* + +Discontinue use of the `node.local_storage` setting. Specifying this setting in +`elasticsearch.yml` will result in an error on startup. +==== + +.The `auth.password` setting for HTTP monitoring has been removed. +[%collapsible] +==== +*Details* + +In Elasticsearch 7.7.0, the setting `xpack.monitoring.exporters..auth.password` +was deprecated in favor of setting `xpack.monitoring.exporters..auth.secure_password`. +In Elasticsearch 8.0.0, the setting `xpack.monitoring.exporters..auth.password` is +removed. + +*Impact* + +Use the `xpack.monitoring.exporters..auth.secure_password` +setting. Discontinue use of the +`xpack.monitoring.exporters..auth.password` setting. Specifying +the `xpack.monitoring.exporters..auth.password` setting in +`elasticsearch.yml` will result in an error on startup. +==== + +.Settings used to disable basic license features have been removed. +[%collapsible] +==== +*Details* + +The following settings were deprecated in {es} 7.8.0 and have been removed +in {es} 8.0.0: + +* `xpack.enrich.enabled` +* `xpack.flattened.enabled` +* `xpack.ilm.enabled` +* `xpack.monitoring.enabled` +* `xpack.rollup.enabled` +* `xpack.slm.enabled` +* `xpack.sql.enabled` +* `xpack.transform.enabled` +* `xpack.vectors.enabled` + +These basic license features are now always enabled. + +If you have disabled ILM so that you can use another tool to manage Watcher +indices, the newly introduced `xpack.watcher.use_ilm_index_management` setting +may be set to false. + +*Impact* + +Discontinue use of the removed settings. Specifying these settings in +`elasticsearch.yml` will result in an error on startup. +==== + +.Settings used to defer cluster recovery pending a certain number of master nodes have been removed. +[%collapsible] +==== +*Details* + +The following cluster settings have been removed: + +* `gateway.expected_nodes` +* `gateway.expected_master_nodes` +* `gateway.recover_after_nodes` +* `gateway.recover_after_master_nodes` + +It is safe to recover the cluster as soon as a majority of master-eligible +nodes have joined so there is no benefit in waiting for any additional +master-eligible nodes to start. + +*Impact* + +Discontinue use of the removed settings. If needed, use +`gateway.expected_data_nodes` or `gateway.recover_after_data_nodes` to defer +cluster recovery pending a certain number of data nodes. +==== + +.Legacy role settings have been removed. +[%collapsible] +==== +*Details* + +The legacy role settings: + +* `node.data` +* `node.ingest` +* `node.master` +* `node.ml` +* `node.remote_cluster_client` +* `node.transform` +* `node.voting_only` + +have been removed. Instead, use the `node.roles` setting. If you were previously +using the legacy role settings on a 7.13 or later cluster, you will have a +deprecation log message on each of your nodes indicating the exact replacement +value for `node.roles`. + +*Impact* + +Discontinue use of the removed settings. Specifying these settings in +`elasticsearch.yml` will result in an error on startup. +==== + +[[system-call-filter-setting]] +.The system call filter setting has been removed. +[%collapsible] +==== +*Details* + +Elasticsearch uses system call filters to remove its ability to fork another +process. This is useful to mitigate remote code exploits. These system call +filters are enabled by default, and were previously controlled via the setting +`bootstrap.system_call_filter`. Starting in Elasticsearch 8.0, system call +filters will be required. As such, the setting `bootstrap.system_call_filter` +was deprecated in Elasticsearch 7.13.0, and is removed as of Elasticsearch +8.0.0. + +*Impact* + +Discontinue use of the removed setting. Specifying this setting in Elasticsearch +configuration will result in an error on startup. +==== + +[[tier-filter-setting]] +.Tier filtering settings have been removed. +[%collapsible] +==== +*Details* + +The cluster and index level settings ending in `._tier` used for filtering the allocation of a shard +to a particular set of nodes have been removed. Instead, the +{ref}/data-tier-shard-filtering.html#tier-preference-allocation-filter[tier +preference setting], `index.routing.allocation.include._tier_preference` should +be used. The removed settings are: + +Cluster level settings: + +- `cluster.routing.allocation.include._tier` +- `cluster.routing.allocation.exclude._tier` +- `cluster.routing.allocation.require._tier` + +Index settings: + +- `index.routing.allocation.include._tier` +- `index.routing.allocation.exclude._tier` +- `index.routing.allocation.require._tier` + +*Impact* + +Discontinue use of the removed settings. Specifying any of these cluster settings in Elasticsearch +configuration will result in an error on startup. Any indices using these settings will have the +settings archived (and they will have no effect) when the index metadata is loaded. +==== + +[[shared-data-path-setting]] +.Shared data path and per index data path settings are deprecated. +[%collapsible] +==== +*Details* + +Elasticsearch uses the shared data path as the base path of per index data +paths. This feature was previously used with shared replicas. Starting in +7.13.0, these settings are deprecated. Starting in 8.0 only existing +indices created in 7.x will be capable of using the shared data path and +per index data path settings. + +*Impact* + +Discontinue use of the deprecated settings. +==== + +[[single-data-node-watermark-setting]] +.The single data node watermark setting is deprecated and now only accepts `true`. +[%collapsible] +==== +*Details* + +In 7.14, setting `cluster.routing.allocation.disk.watermark.enable_for_single_data_node` +to false was deprecated. Starting in 8.0, the only legal value will be +true. In a future release, the setting will be removed completely, with same +behavior as if the setting was `true`. + +If the old behavior is desired for a single data node cluster, disk based +allocation can be disabled by setting +`cluster.routing.allocation.disk.threshold_enabled: false` + +*Impact* + +Discontinue use of the deprecated setting. +==== + +[[auto-import-dangling-indices-removed]] +.The `gateway.auto_import_dangling_indices` setting has been removed. +[%collapsible] +==== +*Details* + +The `gateway.auto_import_dangling_indices` cluster setting has been removed. +Previously, you could use this setting to automatically import +{ref}/modules-gateway.html#dangling-indices[dangling indices]. However, +automatically importing dangling indices is unsafe. Use the +{ref}/indices.html#dangling-indices-api[dangling indices APIs] to manage and +import dangling indices instead. + +*Impact* + +Discontinue use of the removed setting. Specifying the setting in +`elasticsearch.yml` will result in an error on startup. +==== + +.The `listener` thread pool has been removed. +[%collapsible] +==== +*Details* + +Previously, the transport client used the thread pool to ensure listeners aren't +called back on network threads. The transport client has been removed +in 8.0, and the thread pool is no longer needed. + +*Impact* + +Remove `listener` thread pool settings from `elasticsearch.yml` for any nodes. +Specifying `listener` thread pool settings in `elasticsearch.yml` will result in +an error on startup. +==== + +.The `fixed_auto_queue_size` thread pool type has been removed. +[%collapsible] +==== +*Details* + +The `fixed_auto_queue_size` thread pool type, previously marked as an +experimental feature, was deprecated in 7.x and has been removed in 8.0. +The `search` and `search_throttled` thread pools have the `fixed` type now. + +*Impact* + +No action needed. +==== + +.Several `transport` settings have been replaced. +[%collapsible] +==== +*Details* + +The following settings have been deprecated in 7.x and removed in 8.0. Each setting has a replacement +setting that was introduced in 6.7. + +- `transport.tcp.port` replaced by `transport.port` +- `transport.tcp.compress` replaced by `transport.compress` +- `transport.tcp.connect_timeout` replaced by `transport.connect_timeout` +- `transport.tcp_no_delay` replaced by `transport.tcp.no_delay` +- `transport.profiles.profile_name.tcp_no_delay` replaced by `transport.profiles.profile_name.tcp.no_delay` +- `transport.profiles.profile_name.tcp_keep_alive` replaced by `transport.profiles.profile_name.tcp.keep_alive` +- `transport.profiles.profile_name.reuse_address` replaced by `transport.profiles.profile_name.tcp.reuse_address` +- `transport.profiles.profile_name.send_buffer_size` replaced by `transport.profiles.profile_name.tcp.send_buffer_size` +- `transport.profiles.profile_name.receive_buffer_size` replaced by `transport.profiles.profile_name.tcp.receive_buffer_size` + +*Impact* + +Use the replacement settings. Discontinue use of the removed settings. +Specifying the removed settings in `elasticsearch.yml` will result in an error +on startup. +==== + +.Selective transport compression has been enabled by default. +[%collapsible] +==== +*Details* + +Prior to 8.0, transport compression was disabled by default. Starting in 8.0, +`transport.compress` defaults to `indexing_data`. This configuration means that +the propagation of raw indexing data will be compressed between nodes. + +*Impact* + +Inter-node transit will get reduced along the indexing path. In some scenarios, +CPU usage could increase. +==== + +.Transport compression defaults to lz4. +[%collapsible] +==== +*Details* + +Prior to 8.0, the `transport.compression_scheme` setting defaulted to `deflate`. Starting in +8.0, `transport.compress_scheme` defaults to `lz4`. + +Prior to 8.0, the `cluster.remote..transport.compression_scheme` +setting defaulted to `deflate` when `cluster.remote..transport.compress` +was explicitly configured. Starting in 8.0, +`cluster.remote..transport.compression_scheme` will fallback to +`transport.compression_scheme` by default. + +*Impact* + +This configuration means that transport compression will produce somewhat lower +compression ratios in exchange for lower CPU load. +==== +//end::notable-breaking-changes[] + +// This change is not notable because it should not have any impact on upgrades +// However we document it here out of an abundance of caution +[[fips-default-hash-changed]] +.When FIPS mode is enabled the default password hash is now PBKDF2_STRETCH +[%collapsible] +==== +*Details* + +If `xpack.security.fips_mode.enabled` is true (see <>), +the value of `xpack.security.authc.password_hashing.algorithm` now defaults to +`pbkdf2_stretch`. + +In earlier versions this setting would always default to `bcrypt` and a runtime +check would prevent a node from starting unless the value was explicitly set to +a "pbkdf2" variant. + +There is no change for clusters that do not enable FIPS 140 mode. + +*Impact* + +This change should not have any impact on upgraded nodes. +Any node with an explicitly configured value for the password hashing algorithm +will continue to use that configured value. +Any node that did not have an explicitly configured password hashing algorithm in +{es} 6.x or {es} 7.x would have failed to start. +==== diff --git a/docs/reference/migration/migrate_8_0/_command-line-tool-changes.asciidoc b/docs/reference/migration/migrate_8_0/_command-line-tool-changes.asciidoc new file mode 100644 index 0000000000000..94da5c9b6215a --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_command-line-tool-changes.asciidoc @@ -0,0 +1,25 @@ +[discrete] +[[breaking_80_command_line_tool_changes]] +==== Command line tool changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +TIP: {ess-skip-section} + +[[migrate-tool-removed]] +.The `elasticsearch-migrate` tool has been removed. +[%collapsible] +==== +*Details* + +The `elasticsearch-migrate` tool provided a way to convert file +realm users and roles into the native realm. It has been deprecated +since {es} 7.2.0. Users and roles should now be created in the native +realm directly. + +*Impact* + +Discontinue use of the `elasticsearch-migrate` tool. Attempts to use the +`elasticsearch-migrate` tool will result in an error. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_index-setting-changes.asciidoc b/docs/reference/migration/migrate_8_0/_index-setting-changes.asciidoc new file mode 100644 index 0000000000000..4bfa552e31e19 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_index-setting-changes.asciidoc @@ -0,0 +1,74 @@ +[discrete] +[[breaking_80_index_setting_changes]] +==== Index setting changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +[[index-max-adjacency-matrix-filters-removed]] +.The `index.max_adjacency_matrix_filters` index setting has been removed. +[%collapsible] +==== +*Details* + +The `index.max_adjacency_matrix_filters` index setting has been removed. +Previously, you could use this setting to configure the maximum number of +filters for the +{ref}/search-aggregations-bucket-adjacency-matrix-aggregation.html[adjacency +matrix aggregation]. The `indices.query.bool.max_clause_count` index setting now +determines the maximum number of filters for the aggregation. + +*Impact* + +Discontinue use of the `index.max_adjacency_matrix_filters` index setting. + +Requests that include the index setting will return an error. If you upgrade a +cluster with a 7.x index that already contains the setting, {es} will +{ref}/archived-settings.html#archived-index-settings[archive the setting]. + +Remove the index setting from index and component templates. Attempts to use a +template that contains the setting will fail and return an error. This includes +automated operations, such the {ilm-init} rollover action. +==== + +.The `index.force_memory_term_dictionary` setting has been removed. +[%collapsible] +==== +*Details* + +The `index.force_memory_term_dictionary` setting was introduced in 7.0 as a +temporary measure to allow users to opt-out of the optimization that leaves the +term dictionary on disk when appropriate. This optimization is now mandatory +and the setting is removed. + +*Impact* + +Discontinue use of the `index.force_memory_term_dictionary` index setting. +Requests that include this setting will return an error. +==== + +.The `index.soft_deletes.enabled` setting has been removed. +[%collapsible] +==== +*Details* + +Creating indices with soft deletes disabled was deprecated in 7.6 and +is no longer supported in 8.0. The `index.soft_deletes.enabled` setting +can no longer be set to `false`. + +*Impact* + +Discontinue use of the `index.soft_deletes.enabled` index setting. Requests that +set `index.soft_deletes.enabled` to `false` will return an error. +==== + +.The `index.translog.retention.age` and `index.translog.retention.size` settings have been removed. +[%collapsible] +==== +*Details* + +Translog retention settings `index.translog.retention.age` and +`index.translog.retention.size` were effectively ignored in 7.4, deprecated in +7.7, and removed in 8.0 in favor of +{ref}/index-modules-history-retention.html[soft deletes]. + +*Impact* + +Discontinue use of the `index.translog.retention.age` and +`index.translog.retention.size` index settings. Requests that +include these settings will return an error. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_java-api-changes.asciidoc b/docs/reference/migration/migrate_8_0/_java-api-changes.asciidoc new file mode 100644 index 0000000000000..5727f5ef92eca --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_java-api-changes.asciidoc @@ -0,0 +1,55 @@ +[discrete] +[[breaking_80_java_api_changes]] +==== Java API changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +[[ilm-hlrc-rename]] +.The `indexlifecycle` package has been renamed `ilm` in the Java High Level REST Client. +[%collapsible] +==== +*Details* + +In the high level REST client, the `indexlifecycle` package has been +renamed to `ilm` to match the package rename inside the {es} code. + +*Impact* + +Update your workflow and applications to use the `ilm` package in place of +`indexlifecycle`. +==== + +.Changes to `Fuzziness`. +[%collapsible] +==== +*Details* + +To create `Fuzziness` instances, use the `fromString` and `fromEdits` method +instead of the `build` method that used to accept both Strings and numeric +values. Several fuzziness setters on query builders (e.g. +MatchQueryBuilder#fuzziness) now accept only a `Fuzziness`instance instead of +an Object. + +Fuzziness used to be lenient when it comes to parsing arbitrary numeric values +while silently truncating them to one of the three allowed edit distances 0, 1 +or 2. This leniency is now removed and the class will throw errors when trying +to construct an instance with another value (e.g. floats like 1.3 used to get +accepted but truncated to 1). + +*Impact* + +Use the available constants (e.g. `Fuzziness.ONE`, `Fuzziness.AUTO`) or build +your own instance using the above mentioned factory methods. Use only allowed +`Fuzziness` values. +==== + +.Changes to `Repository`. +[%collapsible] +==== +*Details* + +Repository has no dependency on IndexShard anymore. The contract of restoreShard +and snapshotShard has been reduced to Store and MappingService in order to improve +testability. + +*Impact* + +No action needed. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_jvm-option-changes.asciidoc b/docs/reference/migration/migrate_8_0/_jvm-option-changes.asciidoc new file mode 100644 index 0000000000000..8d9c602666f24 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_jvm-option-changes.asciidoc @@ -0,0 +1,59 @@ +[discrete] +[[breaking_80_jvm_option_changes]] +==== JVM option changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +TIP: {ess-skip-section} + +[[breaking_80_allocation_change_flood_stage_block_always_removed]] +.`es.disk.auto_release_flood_stage_block` has been removed. +[%collapsible] +==== +*Details* + +If a node exceeds the flood-stage disk watermark then we add a block to all of +its indices to prevent further writes as a last-ditch attempt to prevent the +node completely exhausting its disk space. By default, from 7.4 onwards the +block is automatically removed when a node drops below the high watermark +again, but this behaviour could be disabled by setting the system property +`es.disk.auto_release_flood_stage_block` to `false`. This behaviour is no +longer optional, and this system property must now not be set. + +*Impact* + +Discontinue use of the `es.disk.auto_release_flood_stage_block` system property. +Setting this system property will result in an error on startup. +==== + +.`es.rest.url_plus_as_space` has been removed. +[%collapsible] +==== +*Details* + +Starting in version 7.4, a `+` in a URL will be encoded as `%2B` by all REST API functionality. Prior versions handled a `+` as a single space. +In these previous versions, if your application required handling `+` as a single space, you could return to the old behaviour by setting the system property +`es.rest.url_plus_as_space` to `true`. Note that this behaviour is deprecated and setting this system property to `true` will cease +to be supported in version 8. + +*Impact* + +Update your application or workflow to assume `+` in a URL is encoded as `%2B`. +==== + +.`es.unsafely_permit_handshake_from_incompatible_builds` has been removed. +[%collapsible] +==== +*Details* + +{es} has a check that verifies that communicating pairs of nodes of the same +version are running exactly the same build and therefore using the same wire +format as each other. In previous versions this check can be bypassed by +setting the system property +`es.unsafely_permit_handshake_from_incompatible_builds` to `true`. The use of +this system property is now forbidden. + +*Impact* + +Discontinue use of the `es.unsafely_permit_handshake_from_incompatible_builds` +system property, and ensure that all nodes of the same version are running +exactly the same build. Setting this system property will result in an error +on startup. +==== +//end::notable-breaking-changes[] \ No newline at end of file diff --git a/docs/reference/migration/migrate_8_0/_logging-changes.asciidoc b/docs/reference/migration/migrate_8_0/_logging-changes.asciidoc new file mode 100644 index 0000000000000..da28aae5f007a --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_logging-changes.asciidoc @@ -0,0 +1,58 @@ +[discrete] +[[breaking_80_logging_changes]] +==== Logging changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +.{es} JSON logs now comply with ECS. +[%collapsible] +==== +*Details* + +{es}'s {ref}/logging.html[JSON logs] now comply with the +{ecs-ref}/index.html[Elastic Common Schema (ECS)]. Previously, {es}'s JSON logs +used a custom schema. + +*Impact* + +If your application parses {es}'s JSON logs, update it to support the new ECS +format. +==== + +.{es} no longer emits deprecation logs or slow logs in plaintext. +[%collapsible] +==== +*Details* + +{es} no longer emits a plaintext version of the following logs: + +* Deprecation logs +* Indexing slow logs +* Search slow logs + +These logs are now only available in JSON. + +Server logs are still available in both a JSON and plaintext format. + +*Impact* + +If your application parses {es}'s plaintext logs, update it to use the new ECS +JSON logs. +==== + +[[audit-logs-are-rolled-over-and-archived-by-size]] +.Audit logs are rolled-over and archived by size. +[%collapsible] +==== +*Details* + +In addition to the existing daily rollover, the security audit logs are +now rolled-over by disk size limit as well. Moreover, the rolled-over logs +are also gzip compressed. + +*Impact* + +The names of rolled over audit log files (but not the name of the current log) +have changed. +If you've set up automated tools to consume these files, you must configure them +to use the new names and to possibly account for `gzip` archives instead of +plain text. The Docker build of {es} is not affected because it logs on `stdout`, +where rollover is not performed. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_mapping-changes.asciidoc b/docs/reference/migration/migrate_8_0/_mapping-changes.asciidoc new file mode 100644 index 0000000000000..404460bb6b957 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_mapping-changes.asciidoc @@ -0,0 +1,138 @@ +[discrete] +[[breaking_80_mapping_changes]] +==== Mapping changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +.Indices created in {es} 6.x and earlier versions are not supported. +[%collapsible] +==== +*Details* + +Elasticsearch 8.0 can read indices created in version 7.0 or above. An +Elasticsearch 8.0 node will not start in the presence of indices created in a +version of Elasticsearch before 7.0. + +*Impact* + +Reindex indices created in {es} 6.x or before with {es} 7.x if they need to be carried forward to {es} 8.x. +==== + +.Closed indices created in {es} 6.x and earlier versions are not supported. +[%collapsible] +==== +*Details* + +In earlier versions a node would start up even if it had data from indices +created in a version before the previous major version, as long as those +indices were closed. {es} now ensures that it is compatible with every index, +open or closed, at startup time. + +*Impact* + +Reindex closed indices created in {es} 6.x or before with {es} 7.x if they need +to be carried forward to {es} 8.x. +==== + +.The maximum number of completion contexts per field is now 10. +[%collapsible] +==== +*Details* + +The number of completion contexts within a single completion field +has been limited to 10. + +*Impact* + +Use a maximum of 10 completion contexts in a completion field. Specifying more +than 10 completion contexts will return an error. +==== + +.Multi-fields within multi-fields is no longer supported. +[%collapsible] +==== +*Details* + +Previously, it was possible to define a multi-field within a multi-field. +Defining chained multi-fields was deprecated in 7.3 and is now no longer +supported. + +*Impact* + +To migrate mappings, all instances of `fields` that occur within +a `fields` block should be removed, either by flattening the chained `fields` +blocks into a single level, or by switching to `copy_to` if appropriate. +==== + +[[fieldnames-enabling]] +.The `_field_names` metadata field's `enabled` parameter has been removed. +[%collapsible] +==== +*Details* + +The setting has been deprecated with 7.5 and is no longer supported on new indices. +Mappings for older indices will continue to work but emit a deprecation warning. + +*Impact* + +The `enabled` setting for `_field_names` should be removed from templates and mappings. +Disabling _field_names is not necessary because it no longer carries a large index overhead. +==== + +[[mapping-boosts]] +.The `boost` parameter on field mappings has been removed. +[%collapsible] +==== +*Details* + +Index-time boosts have been deprecated since the 5x line, but it was still possible +to declare field-specific boosts in the mappings. This is now removed completely. +Indexes built in 7x that contain mapping boosts will emit warnings, and the boosts +will have no effect in 8.0. New indexes will not permit boosts to be set in their +mappings at all. + +*Impact* + +The `boost` setting should be removed from templates and mappings. Use boosts +directly on queries instead. +==== + +.Java-time date formats replace joda-time formats. +[%collapsible] +==== +*Details* + +In 7.0, {es} switched from joda time to java time for date-related parsing, +formatting, and calculations. Indices created in 7.0 and later versions are +already required to use mappings with java-time date formats. However, +earlier indices using joda-time formats must be reindexed to use +mappings with java-time formats. + +*Impact* + +For a detailed migration guide, see the {ref}/migrate-to-java-time.html[Java +time migration guide]. +==== + +[[geo-shape-strategy]] +.Several `geo_shape` mapping parameters have been removed. +[%collapsible] +==== +*Details* + +The following `geo_shape` mapping parameters were deprecated in 6.6: + +* `tree` +* `tree_levels` +* `strategy` +* `distance_error_pct` + +These parameters have been removed in 8.0.0. + +*Impact* + +In 8.0, you can no longer create mappings that include these parameters. +However, 7.x indices that use these mapping parameters will continue to work. +==== + +.The `sparse_vector` field data type has been removed. +[%collapsible] +==== +*Details* + +The `sparse_vector` field type was deprecated in 7.6 and is now removed in +8.0. We have not seen much interest in this experimental field type, and don't +see a clear use case as it's currently designed. If you have feedback or +suggestions around sparse vector functionality, please let us know through +GitHub or the 'discuss' forums. + +*Impact* + +Discontinue use of the `sparse_vector` field data type. Requests containing +a mapping for this field data type will return an error. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_packaging-changes.asciidoc b/docs/reference/migration/migrate_8_0/_packaging-changes.asciidoc new file mode 100644 index 0000000000000..3247d0b723c83 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_packaging-changes.asciidoc @@ -0,0 +1,65 @@ +[discrete] +[[breaking_80_packaging_changes]] +==== Packaging changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +TIP: {ess-skip-section} + +.The layout of the data folder has changed. +[%collapsible] +==== +*Details* + +Each node's data is now stored directly in the data directory set by the +`path.data` setting, rather than in `${path.data}/nodes/0`, because the removal +of the `node.max_local_storage_nodes` setting means that nodes may no longer +share a data path. + +*Impact* + +At startup, {es} will automatically migrate the data path to the new layout. +This automatic migration will not proceed if the data path contains data for +more than one node. You should move to a configuration in which each node has +its own data path before upgrading. + +If you try to upgrade a configuration in which there is data for more than one +node in a data path then the automatic migration will fail and {es} +will refuse to start. To resolve this you will need to perform the migration +manually. The data for the extra nodes are stored in folders named +`${path.data}/nodes/1`, `${path.data}/nodes/2` and so on, and you should move +each of these folders to an appropriate location and then configure the +corresponding node to use this location for its data path. If your nodes each +have more than one data path in their `path.data` settings then you should move +all the corresponding subfolders in parallel. Each node uses the same subfolder +(e.g. `nodes/2`) across all its data paths. +==== + +.The default Maxmind geoip databases have been removed. +[%collapsible] +==== +*Details* + +The default Maxmind geoip databases that shipped by default with Elasticsearch +have been removed. These databases are out dated and stale and using these +databases will likely result in incorrect geoip lookups. + +By default since 7.13, these pre-packaged geoip databases +were used in case no database were specified in the config directory or before +the geoip downloader downloaded the geoip databases. When the geoip database +downloader completed downloading the new databases then these pre-packaged +databases were no longer used. + +*Impact* + +If the geoip downloader is disabled and no geoip databases are provided +in the config directory of each ingest node then the geoip processor will +no longer perform geoip lookups and tag these documents with the fact that +the requested database is no longer available. + +After a cluster has been started and before the geoip downloader has completed +downloading the most up to data databases, the geoip processor will not perform +any geoip lookups and tag documents that the requested database is not available. +After the geoip downloader has completed downloading the most up to data databases +then the geoip processor will function as normal. The window of time that the +geoip processor can't do geoip lookups after cluster startup should be very small. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_painless-changes.asciidoc b/docs/reference/migration/migrate_8_0/_painless-changes.asciidoc new file mode 100644 index 0000000000000..f526002a486d8 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_painless-changes.asciidoc @@ -0,0 +1,46 @@ +[discrete] +[[breaking_80_painless_changes]] +==== Painless changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +.The `JodaCompatibleZonedDateTime` class has been removed. +[%collapsible] +==== +*Details* + +As a transition from Joda datetime to Java datetime, scripting used +an intermediate class called `JodaCompatibleZonedDateTime`. This class +has been removed and is replaced by `ZonedDateTime`. Any use of casting +to a `JodaCompatibleZonedDateTime` or use of method calls only available +in `JodaCompatibleZonedDateTime` in a script will result in a compilation +error, and may not allow the upgraded node to start. + +*Impact* + +Before upgrading, replace `getDayOfWeek` with `getDayOfWeekEnum().value` in any +scripts. Any use of `getDayOfWeek` expecting a return value of `int` will result +in a compilation error or runtime error and may not allow the upgraded node to +start. + +The following `JodaCompatibleZonedDateTime` methods must be replaced using +`ZonedDateTime` methods prior to upgrade: + +* `getMillis()` -> `toInstant().toEpochMilli()` +* `getCenturyOfEra()` -> `get(ChronoField.YEAR_OF_ERA) / 100` +* `getEra()` -> `get(ChronoField.ERA)` +* `getHourOfDay()` -> `getHour()` +* `getMillisOfDay()` -> `get(ChronoField.MILLI_OF_DAY)` +* `getMillisOfSecond()` -> `get(ChronoField.MILLI_OF_SECOND)` +* `getMinuteOfDay()` -> `get(ChronoField.MINUTE_OF_DAY)` +* `getMinuteOfHour()` -> `getMinute()` +* `getMonthOfYear()` -> `getMonthValue()` +* `getSecondOfDay()` -> `get(ChronoField.SECOND_OF_DAY)` +* `getSecondOfMinute()` -> `getSecond()` +* `getWeekOfWeekyear()` -> `get(DateFormatters.WEEK_FIELDS_ROOT.weekBasedYear())` +* `getYearOfCentury()` -> `get(ChronoField.YEAR_OF_ERA) % 100` +* `getYearOfEra()` -> `get(ChronoField.YEAR_OF_ERA)` +* `toString(String)` -> a DateTimeFormatter +* `toString(String, Locale)` -> a DateTimeFormatter +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_rest-api-changes.asciidoc b/docs/reference/migration/migrate_8_0/_rest-api-changes.asciidoc new file mode 100644 index 0000000000000..83a70a7811fb8 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_rest-api-changes.asciidoc @@ -0,0 +1,912 @@ +[discrete] +[[breaking_80_rest_api_changes]] +==== REST API changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +.REST API endpoints containing `_xpack` have been removed. +[%collapsible] +==== +*Details* + +In 7.0, we deprecated REST endpoints that contain `_xpack` in their path. These +endpoints are now removed in 8.0. Each endpoint that was deprecated and removed +is replaced with a new endpoint that does not contain `_xpack`. As an example, +`/{index}/_xpack/graph/_explore` is replaced by `/{index}/_graph/explore`. + +*Impact* + +Use the replacement REST API endpoints. Requests submitted to the `_xpack` +API endpoints will return an error. +==== + +[[remove-term-order-key]] +.The `terms` aggregation no longer supports the `_term` order key. +[%collapsible] +==== +*Details* + +The `terms` aggregation no longer supports the `_term` key in `order` values. To +sort buckets by their term, use `_key` instead. + +*Impact* + +Discontinue use of the `_term` order key. Requests that include a `_term` order +key will return an error. +==== + +[[remove-time-order-key]] +.The `date_histogram` aggregation no longer supports the `_time` order key. +[%collapsible] +==== +*Details* + +The `date_histogram` aggregation no longer supports the `_time` key in `order` +values. To sort buckets by their key, use `_key` instead. + +*Impact* + +Discontinue use of the `_time` order key. Requests that include a `_time` order +key will return an error. +==== + +[[remove-moving-avg-agg]] +.The `moving_avg` aggregation has been removed. +[%collapsible] +==== +*Details* + +The `moving_avg` aggregation was deprecated in 6.4 and has been removed. To +calculate moving averages, use the +{ref}/search-aggregations-pipeline-movfn-aggregation.html[`moving_fn` +aggregation] instead. + +*Impact* + +Discontinue use of the `moving_avg` aggregation. Requests that include the +`moving_avg` aggregation will return an error. +==== + +[[percentile-duplication]] +.The `percentiles` aggregation's `percents` parameter no longer supports duplicate values. +[%collapsible] +==== +*Details* + +If you specify the `percents` parameter with the +{ref}/search-aggregations-metrics-percentile-aggregation.html[`percentiles` aggregation], +its values must be unique. Otherwise, an exception occurs. + +*Impact* + +Use unique values in the `percents` parameter of the `percentiles` aggregation. +Requests containing duplicate values in the `percents` parameter will return +an error. +==== + +[[date-histogram-interval]] +.The `date_histogram` aggregation's `interval` parameter is no longer valid. +[%collapsible] +==== +*Details* + +It is now an error to specify the `interval` parameter to the +{ref}/search-aggregations-bucket-datehistogram-aggregation.html[`date_histogram` +aggregation] or the +{ref}/search-aggregations-bucket-composite-aggregation.html#_date_histogram[`composite +date_histogram` source. Instead, please use either `calendar_interval` or +`fixed_interval` as appropriate. + +*Impact* + +Uses of the `interval` parameter in either the `date_histogram` aggregation or +the `date_histogram` composite source will now generate an error. Instead +please use the more specific `fixed_interval` or `calendar_interval` +parameters. +==== + +[[ngram-edgengram-filter-names-removed]] +.The `nGram` and `edgeNGram` token filter names have been removed. +[%collapsible] +==== +*Details* + +The `nGram` and `edgeNGram` token filter names that have been deprecated since +version 6.4 have been removed. Both token filters can only be used by their +alternative names `ngram` and `edge_ngram` since version 7.0. + +*Impact* + +Use the equivalent `ngram` and `edge_ngram` token filters. Requests containing +the `nGram` and `edgeNGram` token filter names will return an error. +==== + +[[nGram-edgeNGram-tokenizer-dreprecation]] +.The `nGram` and `edgeNGram` tokenizer names have been removed. +[%collapsible] +==== +*Details* + +The `nGram` and `edgeNGram` tokenizer names haven been deprecated with 7.6 and are no longer +supported on new indices. Mappings for indices created after 7.6 will continue to work but +emit a deprecation warning. The tokenizer name should be changed to the fully equivalent +`ngram` or `edge_ngram` names for new indices and in index templates. + +*Impact* + +Use the `ngram` and `edge_ngram` tokenizers. Requests to create new indices +using the `nGram` and `edgeNGram` tokenizer names will return an error. +==== + +.The `in_flight_requests` stat has been renamed `inflight_requests` in logs and diagnostic APIs. +[%collapsible] +==== +*Details* + +The name of the in flight requests circuit breaker in log output and diagnostic APIs (such as the node stats API) changes from `in_flight_requests` to `inflight_requests` to align it with the name of the corresponding settings. + +*Impact* + +Update your workflow and applications to use the `inflight_requests` stat in +place of `in_flight_requests`. +==== + +.The voting configuration exclusions API endpoint has changed. +[%collapsible] +==== +*Details* + +The `POST /_cluster/voting_config_exclusions/{node_filter}` API has been +removed in favour of `POST /_cluster/voting_config_exclusions?node_names=...` +and `POST /_cluster/voting_config_exclusions?node_ids=...` which allow you to +specify the names or IDs of the nodes to exclude. + +*Impact* + +Use `POST /_cluster/voting_config_exclusions?node_ids=...` and specify the nodes +to exclude instead of using a node filter. Requests submitted to the +`/_cluster/voting_config_exclusions/{node_filter}` endpoint will return an +error. +==== + +.Remote system indices are not followed automatically if they match an auto-follow pattern. +[%collapsible] +==== +*Details* + +Remote system indices matching an {ref}/ccr-auto-follow.html[auto-follow +pattern] won't be configured as a follower index automatically. + +*Impact* + +Explicitly {ref}/ccr-put-follow.html[create a follower index] to follow a remote +system index if that's the wanted behaviour. +==== + +.The EQL `wildcard` function has been removed. +[%collapsible] +==== +*Details* + +The `wildcard` function was deprecated in {es} 7.13.0 and has been removed. + +*Impact* + +Use the `like` or `regex` {ref}/eql-syntax.html#eql-syntax-pattern-comparison-keywords[keywords] instead. +==== + +[[ilm-freeze-noop]] +.The ILM `freeze` action is now a no-op. +[%collapsible] +==== +*Details* + +The ILM freeze action is now a no-op and performs no action on the index, as the freeze API endpoint +has been removed in 8.0. + +*Impact* + +Update your ILM policies to remove the `freeze` action from the `cold` phase. +==== + +[[ilm-policy-validation]] +.Additional validation for ILM policies. +[%collapsible] +==== +*Details* + +Creating or updating an ILM policy now requires that any referenced snapshot repositories and SLM +policies exist. + +*Impact* + +Update your code or configuration management to ensure that repositories and SLM policies are created +before any policies that reference them. +==== + +.The deprecated `_upgrade` API has been removed. +[%collapsible] +==== +*Details* + +Previously, the `_upgrade` API upgraded indices from the previous major +version to the current version. The `_reindex` API should be used +instead for that purpose. + +*Impact* + +Requests made to the old `_upgrade` API will return an error. +==== + +.The deprecated freeze index API has been removed. +[%collapsible] +==== +*Details* + +The freeze index API (`POST //_freeze`) has been removed. +https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage[Improvements +in heap memory usage] have eliminated the reason to freeze indices. +You can still unfreeze existing frozen indices using the +{ref}/unfreeze-index-api.html[unfreeze index API]. For some use cases, the +frozen tier may be a suitable replacement for frozen indices. See +{ref}/data-tiers.html[data tiers] for more information. + +*Impact* + +Requests made to the old freeze index API will return an error. +==== + +.The force merge API's `max_num_segments` and `only_expunge_deletes` parameters cannot both be specified in the same request. +[%collapsible] +==== +*Details* + +Previously, the force merge API allowed the parameters `only_expunge_deletes` +and `max_num_segments` to be set to a non default value at the same time. But +the `max_num_segments` was silently ignored when `only_expunge_deletes` is set +to `true`, leaving the false impression that it has been applied. + +*Impact* + +When using the {ref}/indices-forcemerge.html[force merge API], do not specify +values for both the `max_num_segments` and `only_expunge_deletes` parameters. +Requests that include values for both parameters will return an error. +==== + +.The create or update index template API's `template` parameter has been removed. +[%collapsible] +==== +*Details* + +In 6.0, we deprecated the `template` parameter in create or update index +template requests in favor of using `index_patterns`. Support for the `template` +parameter is now removed in 8.0. + +*Impact* + +Use the {ref}/indices-templates-v1.html[create or update index template API]'s +`index_patterns` parameter. Requests that include the `template` parameter will +return an error. +==== + +.Synced flush has been removed. +[%collapsible] +==== +*Details* + +Synced flush was deprecated in 7.6 and is removed in 8.0. Use a regular flush +instead as it has the same effect as a synced flush in 7.6 and later. + +*Impact* + +Use the {ref}/indices-flush.html[flush API]. Requests to the +`//flush/synced` or `/flush/synced` endpoints will return an error. +==== + +.The default for the `?wait_for_active_shards` parameter on the close index API has changed. +[%collapsible] +==== +*Details* + +When closing an index in earlier versions, by default {es} would not wait for +the shards of the closed index to be properly assigned before returning. From +version 8.0 onwards the default behaviour is to wait for shards to be assigned +according to the +{ref}/docs-index_.html#index-wait-for-active-shards[`index.write.wait_for_active_shards` +index setting]. + +*Impact* + +Accept the new behaviour, or specify `?wait_for_active_shards=0` to preserve +the old behaviour if needed. +==== + +.The index stats API's `types` query parameter has been removed. +[%collapsible] +==== +*Details* + +The index stats API's `types` query parameter has been removed. Previously, you +could combine `types` with the `indexing` query parameter to return indexing +stats for specific mapping types. Mapping types have been removed in 8.0. + +*Impact* + +Discontinue use of the `types` query parameter. Requests that include the +parameter will return an error. +==== + +.The `user_agent` ingest processor's `ecs` parameter has no effect. +[%collapsible] +==== +*Details* + +In 7.2, we deprecated the `ecs` parameter for the `user_agent` ingest processor. +In 8.x, the `user_agent` ingest processor will only return {ecs-ref}[Elastic +Common Schema (ECS)] fields, regardless of the `ecs` value. + +*Impact* + +To avoid deprecation warnings, remove the parameter from your ingest pipelines. +If a pipeline specifies an `ecs` value, the value is ignored. +==== + +.Mapping API endpoints containing mapping types have been removed. +[%collapsible] +==== +*Details* + +The typed REST endpoints of the update mapping, get mapping and get field mapping +APIs have been removed in favour of their typeless REST endpoints, since indexes +no longer contain types, these typed endpoints are obsolete. + +*Impact* + +Use the typeless REST endpoints to update and retrieve mappings. Requests +submitted to the typed mapping API endpoints will return an error. +==== + +.The `include_type_name` query parameter has been removed. +[%collapsible] +==== +*Details* + +The `include_type_name` query parameter has been removed from the index +creation, index template, and mapping APIs. Previously, you could set +`include_type_name` to `true` to indicate that requests and responses should +include a mapping type name. Mapping types have been removed in 8.x. + +*Impact* + +Discontinue use of the `include_type_name` query parameter. Requests that +include the parameter will return an error. +==== + +.Reindex from remote now re-encodes URL-encoded index names. +[%collapsible] +==== +*Details* + +Reindex from remote would previously allow URL-encoded index names and not +re-encode them when generating the search request for the remote host. This +leniency has been removed such that all index names are correctly encoded when +reindex generates remote search requests. + +*Impact* + +Specify unencoded index names for reindex from remote requests. +==== + +.Reindex-related REST API endpoints containing mapping types have been removed. +[%collapsible] +==== +*Details* + +The `/{index}/{type}/_delete_by_query` and `/{index}/{type}/_update_by_query` REST endpoints have been removed in favour of `/{index}/_delete_by_query` and `/{index}/_update_by_query`, since indexes no longer contain types, these typed endpoints are obsolete. + +*Impact* + +Use the replacement REST API endpoints. Requests submitted to API endpoints +that contain a mapping type will return an error. +==== + +.In the reindex, delete by query, and update by query APIs, the `size` parameter has been renamed. +[%collapsible] +==== +*Details* + +Previously, a `_reindex` request had two different size specifications in the body: + +- Outer level, determining the maximum number of documents to process +- Inside the `source` element, determining the scroll/batch size. + +The outer level `size` parameter has now been renamed to `max_docs` to +avoid confusion and clarify its semantics. + +Similarly, the `size` parameter has been renamed to `max_docs` for +`_delete_by_query` and `_update_by_query` to keep the 3 interfaces consistent. + +*Impact* + +Use the replacement parameters. Requests containing the `size` parameter will +return an error. +==== + +.The update by query API now rejects unsupported `script` fields. +[%collapsible] +==== +*Details* + +An update by query API request that includes an unsupported field in the +`script` object now returns an error. Previously, the API would silently ignore +these unsupported fields. + +*Impact* + +To avoid errors, remove unsupported fields from the `script` object. +==== + +.The cat node API's `local` query parameter has been removed. +[%collapsible] +==== +*Details* + +The `?local` parameter to the `GET _cat/nodes` API was deprecated in 7.x and is +rejected in 8.0. This parameter caused the API to use the local cluster state +to determine the nodes returned by the API rather than the cluster state from +the master, but this API requests information from each selected node +regardless of the `?local` parameter which means this API does not run in a +fully node-local fashion. + +*Impact* + +Discontinue use of the `?local` query parameter. {ref}/cat-nodes.html[cat node +API] requests that include this parameter will return an error. +==== + +.The cat shard API's `local` query parameter has been removed. +[%collapsible] +==== +*Details* + +The `?local` parameter to the `GET _cat/shards` API was deprecated in 7.x and is +rejected in 8.0. This parameter caused the API to use the local cluster state +to determine the nodes returned by the API rather than the cluster state from +the master, but this API requests information from each selected node +regardless of the `?local` parameter which means this API does not run in a +fully node-local fashion. + +*Impact* + +Discontinue use of the `?local` query parameter. {ref}/cat-shards.html[cat shards +API] requests that include this parameter will return an error. +==== + +.The cat indices API's `local` query parameter has been removed. +[%collapsible] +==== +*Details* + +The `?local` parameter to the `GET _cat/indices` API was deprecated in 7.x and is +rejected in 8.0. This parameter caused the API to use the local cluster state +to determine the nodes returned by the API rather than the cluster state from +the master, but this API requests information from each selected node +regardless of the `?local` parameter which means this API does not run in a +fully node-local fashion. + +*Impact* + +Discontinue use of the `?local` query parameter. {ref}/cat-indices.html[cat indices +API] requests that include this parameter will return an error. +==== + +.The get field mapping API's `local` query parameter has been removed. +[%collapsible] +==== +*Details* + +The `local` parameter for get field mapping API was deprecated in 7.8 and is +removed in 8.0. This parameter is a no-op and field mappings are always retrieved +locally. + +*Impact* + +Discontinue use of the `local` query parameter. +{ref}/indices-get-field-mapping.html[get field mapping API] requests that +include this parameter will return an error. +==== + +.Post data to jobs API is deprecated. +[%collapsible] +==== +*Details* + +The {ml} {ref}/ml-post-data.html[post data to jobs API] is deprecated starting in 7.11.0 +and will be removed in a future major version. + +*Impact* + +Use {ref}/ml-apis.html#ml-api-datafeed-endpoint[{dfeeds}] instead. +==== + +.The `job_id` property of the Update {dfeeds} API has been removed. +[%collapsible] +==== +*Details* + +The ability to update a `job_id` in a {dfeed} was deprecated in 7.3.0. and is +removed in 8.0. + +*Impact* + +It is not possible to move {dfeeds} between {anomaly-jobs}. +==== + +.Create repository and delete repository API's return `409` status code when a repository is in use instead of `500`. +[%collapsible] +==== +*Details* + +The {ref}/put-snapshot-repo-api.html[Create or update snapshot repository API] and +{ref}/delete-snapshot-repo-api.html[Delete snapshot repository API] return `409` +status code when the request is attempting to modify an existing repository that's in use instead of status code `500`. + +*Impact* + +Update client code that handles creation and deletion of repositories to reflect this change. +==== + +.The `allow_no_datafeeds` property has been removed from {ml} APIs. +[%collapsible] +==== +*Details* + +The `allow_no_datafeeds` property was deprecated in the +{ref}/cat-datafeeds.html[cat {dfeeds}], +{ref}/ml-get-datafeed.html[get {dfeeds}], +{ref}/ml-get-datafeed-stats.html[get {dfeed} statistics], and +{ref}/ml-stop-datafeed.html[stop {dfeeds}] APIs in 7.10.0. + +*Impact* + +Use `allow_no_match` instead. +==== + +.The `allow_no_jobs` property has been removed from {ml} APIs. +[%collapsible] +==== +*Details* + +The `allow_no_jobs` property was deprecated in the +{ref}/cat-anomaly-detectors.html[cat anomaly detectors], +{ref}/ml-close-job.html[close {anomaly-jobs}], +{ref}/ml-get-job.html[get {anomaly-jobs}], +{ref}/ml-get-job-stats.html[get {anomaly-job} statistics], and +{ref}/ml-get-overall-buckets.html[get overall buckets] APIs in 7.10.0. + +*Impact* + +Use `allow_no_match` instead. +==== + +.The StartRollupJob endpoint now returns a success status if a job has already started. +[%collapsible] +==== +*Details* + +Previously, attempting to start an already-started rollup job would +result in a `500 InternalServerError: Cannot start task for Rollup Job +[job] because state was [STARTED]` exception. + +Now, attempting to start a job that is already started will just +return a successful `200 OK: started` response. + +*Impact* + +Update your workflow and applications to assume that a 200 status in response to +attempting to start a rollup job means the job is in an actively started state. +The request itself may have started the job, or it was previously running and so +the request had no effect. +==== + +.Stored scripts no longer support empty scripts or search templates. +[%collapsible] +==== +*Details* + +The {ref}/create-stored-script-api.html[create or update stored script API]'s +`source` parameter cannot be empty. + +*Impact* + +Before upgrading, use the {ref}/delete-stored-script-api.html[delete stored +script API] to delete any empty stored scripts or search templates. +In 8.0, {es} will drop any empty stored scripts or empty search templates from +the cluster state. Requests to create a stored script or search template with +an empty `source` will return an error. +==== + +.The create or update stored script API's `code` parameter has been removed. +[%collapsible] +==== +*Details* + +The {ref}/create-stored-script-api.html[create or update stored script API]'s +`code` parameter has been removed. Use the `source` parameter instead. + +*Impact* + +Discontinue use of the `code` parameter. Requests that include the parameter +will return an error. +==== + +[[_type-search-matches-no-docs]] +.Searches on the `_type` field are no longer supported. +[%collapsible] +==== +*Details* + +In 8.x, the `_type` metadata field has been removed. {es} now handles a search +on the `_type` field as a search on a non-existent field. A search on a +non-existent field matches no documents, regardless of the query string. + +In 7.x, a search for `_doc` in the `_type` field would match the same documents +as a `match_all` query. + +*Impact* + +Remove queries on the `_type` field from your search requests and search +templates. Searches that include these queries may return no results. +==== + +[[msearch-empty-line-support]] +.The multi search API now parses an empty first line as action metadata in text files. +[%collapsible] +==== +*Details* + +The multi search API now parses an empty first line as empty action metadata +when you provide a text file as the request body, such as when using curl's +`--data-binary` flag. + +The API no longer supports text files that contain: + +* An empty first line followed by a line containing only `{}`. +* An empty first line followed by another empty line. + +*Impact* + +Don't provide an unsupported text file to the multi search API. Requests that +include an unsupported file will return an error. +==== + +[[remove-unmapped-type-string]] +.The `unmapped_type: string` sort option has been removed. +[%collapsible] +==== +*Details* + +Search requests no longer support the `unmapped_type: string` sort option. +Instead, use `unmapped_type: keyword` to handle an unmapped field as if it had +the `keyword` field type but ignore its values for sorting. + +*Impact* + +Discontinue use of `unmapped_type: string`. Search requests that include the +`unmapped_type: string` sort option will return shard failures. +==== + +[[id-field-data]] +.Aggregating and sorting on `_id` is disallowed by default. +[%collapsible] +==== +*Details* + +Previously, it was possible to aggregate and sort on the built-in `_id` field +by loading an expensive data structure called fielddata. This was deprecated +in 7.6 and is now disallowed by default in 8.0. + +*Impact* + +Aggregating and sorting on `_id` should be avoided. As an alternative, the +`_id` field's contents can be duplicated into another field with docvalues +enabled (note that this does not apply to auto-generated IDs). +==== + +.Search-related REST API endpoints containing mapping types have been removed. +[%collapsible] +==== +*Details* + +The `/{index}/{type}/_search`, `/{index}/{type}/_msearch`, `/{index}/{type}/_search/template` and `/{index}/{type}/_msearch/template` REST endpoints have been removed in favour of `/{index}/_search`, `/{index}/_msearch`, `/{index}/_search/template` and `/{index}/_msearch/template`; since indexes no longer contain types, these typed endpoints are obsolete.. + +The `/{index}/{type}/_termvectors`, `/{index}/{type}/{id}/_termvectors` and `/{index}/{type}/_mtermvectors` REST endpoints have been removed in favour of `/{index}/_termvectors`, `/{index}/{id}/_termvectors` and `/{index}/_mtermvectors`; since indexes no longer contain types, these typed endpoints are obsolete.. + +The `/{index}/{type}/{doc}` and `/{index}/{type}/_mget` REST endpoints have been removed in favour of `/{index}/_doc/{doc}` and `/{index}/_mget`; since indexes no longer contain types, these typed endpoints are obsolete. + +*Impact* + +Use the replacement REST API endpoints. Requests submitted to API endpoints that +contain a mapping type will return an error. +==== + +.The `common` query has been removed. +[%collapsible] +==== +*Details* + +The `common` query, deprecated in 7.x, has been removed in 8.0. +The same functionality can be achieved by the `match` query if the total number of hits is not tracked. + +*Impact* + +Discontinue use of the `common` query. Search requests containing a `common` +query will return an error. +==== + +.The `cutoff_frequency` parameter has been removed from the `match` and `multi_match` query. +[%collapsible] +==== +*Details* + +The `cutoff_frequency` parameter, deprecated in 7.x, has been removed in 8.0 from `match` and `multi_match` queries. +The same functionality can be achieved without any configuration provided that the total number of hits is not tracked. + +*Impact* + +Discontinue use of the `cutoff_frequency` parameter. Search requests containing +this parameter in a `match` or `multi_match` query will return an error. +==== + +.The `nested_filter` and `nested_path` properties have been removed from the search API's `sort` request body parameter. +[%collapsible] +==== +*Details* + +The `nested_filter` and `nested_path` options, deprecated in 6.x, have been removed in favor of the `nested` context. + +*Impact* + +Discontinue use of the `sort` request body parameter's `nested_filter` and +`nested_path` properties. Requests containing these properties will return an +error. +==== + +.Search and get requests are now routed to shards using adaptive replica selection by default. +[%collapsible] +==== +*Details* + +{es} will no longer prefer using shards in the same location (with the same awareness attribute values) to process +`_search` and `_get` requests. Adaptive replica selection (activated by default in this version) will route requests +more efficiently using the service time of prior inter-node communications. + +*Impact* + +No action needed. +==== + +.Vector functions using `(query, doc['field'])` are no longer supported. +[%collapsible] +==== +*Details* + +The vector functions of the form `function(query, doc['field'])` were +deprecated in 7.6, and are now removed in 8.x. The form +`function(query, 'field')` should be used instead. For example, +`cosineSimilarity(query, doc['field'])` is replaced by +`cosineSimilarity(query, 'field')`. + +*Impact* + +Use the `function(query, 'field')` form. Discontinue use of the `function(query, +doc['field'])` form. Requests containing the `function(query, +doc['field'])` form will return an error. +==== + +.The search API's `indices_boost` request body parameter no longer accepts object values. +[%collapsible] +==== +*Details* + +The `indices_boost` option in the search request used to accept the boosts +both as an object and as an array. The object format has been deprecated since +5.2 and is now removed in 8.0. + +*Impact* + +Use only array values in the `indices_boost` parameter. Requests containing an +object value in the `indices_boost` parameter will return an error. +==== + +.The search API's `use_field_mapping` request body parameter has been removed. +[%collapsible] +==== +*Details* + +In 7.0, we began formatting `docvalue_fields` by default using each field's +mapping definition. To ease the transition from 6.x, we added the format +option `use_field_mapping`. This parameter was deprecated in 7.0, and is now +removed in 8.0. + +*Impact* + +Discontinue use of the `use_field_mapping` request body parameter. Requests +containing this parameter will return an error. +==== + +.The search API's `from` request body and url parameter cannot be negative. +[%collapsible] +==== +*Details* + +Search request used to accept `-1` as a `from` in the search body and the url, +treating it as the default value of 0. Other negative values got rejected with +an error already. We now also reject `-1` as an invalid value. + +*Impact* + +Change any use of `-1` as `from` parameter in request body or url parameters by either +setting it to `0` or omitting it entirely. Requests containing negative values will +return an error. +==== + +.Range queries on date fields treat numeric values alwas as milliseconds-since-epoch. +[%collapsible] +==== +*Details* + +Range queries on date fields used to misinterpret small numbers (e.g. four digits like 1000) +as a year when no additional format was set, but would interpret other numeric values as +milliseconds since epoch. We now treat all numeric values in absence of a specific `format` +parameter as milliseconds since epoch. If you want to query for years instead, with a missing +`format` you now need to quote the input value (e.g. "1984"). + +*Impact* + +If you query date fields without a specified `format`, check if the values in your queries are +actually meant to be milliseconds-since-epoch and use a numeric value in this case. If not, use +a string value which gets parsed by either the date format set on the field in the mappings or +by `strict_date_optional_time` by default. +==== + +.The `geo_bounding_box` query's `type` parameter has been removed. +[%collapsible] +==== +*Details* + +The `geo_bounding_box` query's `type` parameter was deprecated in 7.14.0 and has +been removed in 8.0.0. This parameter is a no-op and has no effect on the query. + +*Impact* + +Discontinue use of the `type` parameter. `geo_bounding_box` queries that include +this parameter will return an error. +==== + +.The `type` query has been removed. +[%collapsible] +==== +*Details* + +The `type` query has been removed. Mapping types have been removed in 8.0. + +*Impact* + +Discontinue use of the `type` query. Requests that include the `type` query +will return an error. +==== + +.The `kibana_user` role has been renamed `kibana_admin`. +[%collapsible] +==== +*Details* + +Users who were previously assigned the `kibana_user` role should instead be assigned +the `kibana_admin` role. This role grants the same set of privileges as `kibana_user`, but has been +renamed to better reflect its intended use. + +*Impact* + +Assign users with the `kibana_user` role to the `kibana_admin` role. +Discontinue use of the `kibana_user` role. +==== + +.The `repositories.fs.compress` node-level setting has been removed. +[%collapsible] +==== +*Details* + +For shared file system repositories (`"type": "fs"`), the node level setting `repositories.fs.compress` could +previously be used to enable compression for all shared file system repositories where `compress` was not specified. +The `repositories.fs.compress` setting has been removed. + +*Impact* + +Use the repository specific `compress` setting to enable compression. See +{ref}/snapshots-register-repository.html[Register a snapshot repository] for +information on the `compress` setting. + +Discontinue use of the `repositories.fs.compress` node-level setting. +==== + +.Snapshots compress metadata files by default. +[%collapsible] +==== +*Details* + +Previously, the default value for `compress` was `false`. The default has been changed to `true`. + +This change will affect both newly created repositories and existing repositories where `compress=false` has not been +explicitly specified. + +For more information on the compress option, see +{ref}/snapshots-register-repository.html[Register a snapshot repository]. + +*Impact* + +Update your workflow and applications to assume a default value of `true` for +the `compress` parameter. +==== + +.The S3 repository plugin now uses a DNS-style access pattern by default. +[%collapsible] +==== +*Details* + +Starting in version 7.4 the `repository-s3` plugin does not use the +now-deprecated path-style access pattern by default. In versions 7.0, 7.1, 7.2 +and 7.3 the `repository-s3` plugin always used the path-style access pattern. +This is a breaking change for deployments that only support path-style access +but which are recognized as supporting DNS-style access by the AWS SDK. This +breaking change was made necessary by +https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/[AWS's +announcement] that the path-style access pattern is deprecated and will be +unsupported on buckets created after September 30th 2020. + +*Impact* + +If your deployment only supports path-style access and is affected by this +change then you must configure the S3 client setting `path_style_access` to +`true`. +==== + +.Restore requests no longer accept settings. +[%collapsible] +==== +*Details* + +In earlier versions, you could pass both `settings` and `index_settings` in the +body of a restore snapshot request, but the `settings` value was ignored. The +restore snapshot API now rejects requests that include a `settings` value. + +*Impact* + +Discontinue use of the `settings` parameter in restore +snapshot request. Requests that include these parameters will return an error. +==== + +.The repository stats API has been removed. +[%collapsible] +==== +*Details* + +The repository stats API has been removed. We deprecated this experimental API +in 7.10.0. + +*Impact* + +Use the {ref}/repositories-metering-apis.html[repositories metering APIs] +instead. +==== + +.Watcher history now writes to a hidden data stream. +[%collapsible] +==== +*Details* + +In 8.x, {es} writes Watcher history to a hidden +`.watcher-history-` data stream. Previously, {es} wrote +Watcher history to hidden +`.watcher-history--` indices. + +*Impact* + +Update your requests to target the Watcher history data stream. For example, use +the `.watcher-history-*` wildcard expression. Requests that specifically target +non-existent Watcher history indices may return an error. +==== + +.HTTP Status code has changed for the Cluster Health API in case of a server timeout. +[%collapsible] +==== +*Details* + +The {ref}/cluster-health.html[cluster health API] includes options for waiting +for certain health conditions to be satisfied. If the requested conditions are +not satisfied within a timeout then {es} will send back a normal response +including the field `"timed_out": true`. In earlier versions it would also use +the HTTP response code `408 Request timeout` if the request timed out, and `200 +OK` otherwise. The `408 Request timeout` response code is not appropriate for +this situation, so from version 8.0.0 {es} will use the response code `200 OK` +for both cases. + +*Impact* + +To detect a server timeout, check the `timed_out` field of the JSON response. +==== +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/_system-req-changes.asciidoc b/docs/reference/migration/migrate_8_0/_system-req-changes.asciidoc new file mode 100644 index 0000000000000..13564b4cb4c75 --- /dev/null +++ b/docs/reference/migration/migrate_8_0/_system-req-changes.asciidoc @@ -0,0 +1,64 @@ +[discrete] +[[breaking_80_system_req_changes]] +==== System requirement changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] +TIP: {ess-skip-section} + +.Several EOL operating systems are no longer supported. +[%collapsible] +==== +*Details* + +The following operating systems have reached their end of life and are no longer +supported by {es}: + +* Amazon Linux +* CentOS 6 +* Debian 8 +* openSUSE Leap 42 +* Oracle Enterprise Linux 6 +* Ubuntu 16.04 + +We've also removed support for `SysV init`. No supported operating systems use +the `SysV init` process. + +*Details* + +Ensure your nodes use a +https://www.elastic.co/support/matrix#matrix_os[supported operating system]. +Running {es} on an unsupported operating system can result in unexpected errors +or failures. +==== + +.Java 17 is required. +[%collapsible] +==== +*Details* + +Java 17 or higher is now required to run {es} and any of its command +line tools. + +*Impact* + +Use Java 17 or higher. Attempts to run {es} 8.0 using earlier Java versions will +fail. + +There is not yet a FIPS-certified security module for Java 17 +that you can use when running {es} 8.0 in FIPS 140-2 mode. +If you run in FIPS 140-2 mode, you will either need to request an exception +from your security organization to upgrade to {es} 8.0, +or remain on {es} 7.x until Java 17 is certified. +==== + +.`JAVA_HOME` is no longer supported. +[%collapsible] +==== +*Details* + +`JAVA_HOME` is no longer supported to set the path for the JDK. Instead, use +the bundled JDK (preferable), or set `ES_JAVA_HOME`. + +*Impact* + +Use the bundled JDK (preferable), or set `ES_JAVA_HOME`. `JAVA_HOME` will be +ignored. +==== +//end::notable-breaking-changes[] \ No newline at end of file diff --git a/docs/reference/migration/migrate_8_0/aggregations.asciidoc b/docs/reference/migration/migrate_8_0/aggregations.asciidoc index b05821e125e97..537bab6c141dd 100644 --- a/docs/reference/migration/migrate_8_0/aggregations.asciidoc +++ b/docs/reference/migration/migrate_8_0/aggregations.asciidoc @@ -4,104 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[index-max-adjacency-matrix-filters-removed]] -.The `index.max_adjacency_matrix_filters` index setting has been removed. -[%collapsible] -==== -*Details* + -The `index.max_adjacency_matrix_filters` index setting has been removed. -Previously, you could use this setting to configure the maximum number of -filters for the -{ref}/search-aggregations-bucket-adjacency-matrix-aggregation.html[adjacency -matrix aggregation]. The `indices.query.bool.max_clause_count` index setting now -determines the maximum number of filters for the aggregation. - -*Impact* + -Discontinue use of the `index.max_adjacency_matrix_filters` index setting. - -Requests that include the index setting will return an error. If you upgrade a -cluster with a 7.x index that already contains the setting, {es} will -{ref}/archived-settings.html#archived-index-settings[archive the setting]. - -Remove the index setting from index and component templates. Attempts to use a -template that contains the setting will fail and return an error. This includes -automated operations, such the {ilm-init} rollover action. -==== - -[[remove-term-order-key]] -.The `terms` aggregation no longer supports the `_term` order key. -[%collapsible] -==== -*Details* + -The `terms` aggregation no longer supports the `_term` key in `order` values. To -sort buckets by their term, use `_key` instead. - -*Impact* + -Discontinue use of the `_term` order key. Requests that include a `_term` order -key will return an error. -==== - -[[remove-time-order-key]] -.The `date_histogram` aggregation no longer supports the `_time` order key. -[%collapsible] -==== -*Details* + -The `date_histogram` aggregation no longer supports the `_time` key in `order` -values. To sort buckets by their key, use `_key` instead. - -*Impact* + -Discontinue use of the `_time` order key. Requests that include a `_time` order -key will return an error. -==== - -[[remove-moving-avg-agg]] -.The `moving_avg` aggregation has been removed. -[%collapsible] -==== -*Details* + -The `moving_avg` aggregation was deprecated in 6.4 and has been removed. To -calculate moving averages, use the -{ref}/search-aggregations-pipeline-movfn-aggregation.html[`moving_fn` -aggregation] instead. - -*Impact* + -Discontinue use of the `moving_avg` aggregation. Requests that include the -`moving_avg` aggregation will return an error. -==== - -[[percentile-duplication]] -.The `percentiles` aggregation's `percents` parameter no longer supports duplicate values. -[%collapsible] -==== -*Details* + -If you specify the `percents` parameter with the -{ref}/search-aggregations-metrics-percentile-aggregation.html[`percentiles` aggregation], -its values must be unique. Otherwise, an exception occurs. - -*Impact* + -Use unique values in the `percents` parameter of the `percentiles` aggregation. -Requests containing duplicate values in the `percents` parameter will return -an error. -==== - -[[date-histogram-interval]] -.The `date_histogram` aggregation's `interval` parameter is no longer valid. -[%collapsible] -==== -*Details* + -It is now an error to specify the `interval` parameter to the -{ref}/search-aggregations-bucket-datehistogram-aggregation.html[`date_histogram` -aggregation] or the -{ref}/search-aggregations-bucket-composite-aggregation.html#_date_histogram[`composite -date_histogram` source. Instead, please use either `calendar_interval` or -`fixed_interval` as appropriate. - -*Impact* + -Uses of the `interval` parameter in either the `date_histogram` aggregation or -the `date_histogram` composite source will now generate an error. Instead -please use the more specific `fixed_interval` or `calendar_interval` -parameters. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/allocation.asciidoc b/docs/reference/migration/migrate_8_0/allocation.asciidoc index b8075a10a5337..b2bb2457ce690 100644 --- a/docs/reference/migration/migrate_8_0/allocation.asciidoc +++ b/docs/reference/migration/migrate_8_0/allocation.asciidoc @@ -4,42 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[breaking_80_allocation_change_flood_stage_block_always_removed]] -.The automatic removal of flood-stage blocks is no longer optional. -[%collapsible] -==== -*Details* + -If a node exceeds the flood-stage disk watermark then we add a block to all of -its indices to prevent further writes as a last-ditch attempt to prevent the -node completely exhausting its disk space. By default, from 7.4 onwards the -block is automatically removed when a node drops below the high watermark -again, but this behaviour could be disabled by setting the system property -`es.disk.auto_release_flood_stage_block` to `false`. This behaviour is no -longer optional, and this system property must now not be set. - -*Impact* + -Discontinue use of the `es.disk.auto_release_flood_stage_block` system property. -Setting this system property will result in an error on startup. -==== - -[[breaking_80_allocation_change_include_relocations_removed]] -.Accounting for the disk usage of relocating shards is no longer optional. -[%collapsible] -==== -*Details* + -By default {es} will account for the sizes of relocating shards when making -allocation decisions based on the disk usage of the nodes in the cluster. In -earlier versions the `cluster.routing.allocation.disk.include_relocations` -setting allowed this accounting to be disabled, which would result in poor -allocation decisions that might overshoot watermarks and require significant -extra work to correct. This behaviour is no longer optional, and this setting -has been removed. - -*Impact* + -Discontinue use of the `cluster.routing.allocation.disk.include_relocations` -setting. Specifying this setting in `elasticsearch.yml` will result in an error -on startup. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/analysis.asciidoc b/docs/reference/migration/migrate_8_0/analysis.asciidoc index 9397dc317d367..e604a98588d80 100644 --- a/docs/reference/migration/migrate_8_0/analysis.asciidoc +++ b/docs/reference/migration/migrate_8_0/analysis.asciidoc @@ -4,34 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[ngram-edgengram-filter-names-removed]] -.The `nGram` and `edgeNGram` token filter names have been removed. -[%collapsible] -==== -*Details* + -The `nGram` and `edgeNGram` token filter names that have been deprecated since -version 6.4 have been removed. Both token filters can only be used by their -alternative names `ngram` and `edge_ngram` since version 7.0. - -*Impact* + -Use the equivalent `ngram` and `edge_ngram` token filters. Requests containing -the `nGram` and `edgeNGram` token filter names will return an error. -==== - -[[nGram-edgeNGram-tokenizer-dreprecation]] -.The `nGram` and `edgeNGram` tokenizer names have been removed. -[%collapsible] -==== -*Details* + -The `nGram` and `edgeNGram` tokenizer names haven been deprecated with 7.6 and are no longer -supported on new indices. Mappings for indices created after 7.6 will continue to work but -emit a deprecation warning. The tokenizer name should be changed to the fully equivalent -`ngram` or `edge_ngram` names for new indices and in index templates. - -*Impact* + -Use the `ngram` and `edge_ngram` tokenizers. Requests to create new indices -using the `nGram` and `edgeNGram` tokenizer names will return an error. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/api.asciidoc b/docs/reference/migration/migrate_8_0/api.asciidoc index 20a142ecda456..0c346a27a6363 100644 --- a/docs/reference/migration/migrate_8_0/api.asciidoc +++ b/docs/reference/migration/migrate_8_0/api.asciidoc @@ -4,130 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The cat node API's `local` query parameter has been removed. -[%collapsible] -==== -*Details* + -The `?local` parameter to the `GET _cat/nodes` API was deprecated in 7.x and is -rejected in 8.0. This parameter caused the API to use the local cluster state -to determine the nodes returned by the API rather than the cluster state from -the master, but this API requests information from each selected node -regardless of the `?local` parameter which means this API does not run in a -fully node-local fashion. - -*Impact* + -Discontinue use of the `?local` query parameter. {ref}/cat-nodes.html[cat node -API] requests that include this parameter will return an error. -==== - -.The cat shard API's `local` query parameter has been removed. -[%collapsible] -==== -*Details* + -The `?local` parameter to the `GET _cat/shards` API was deprecated in 7.x and is -rejected in 8.0. This parameter caused the API to use the local cluster state -to determine the nodes returned by the API rather than the cluster state from -the master, but this API requests information from each selected node -regardless of the `?local` parameter which means this API does not run in a -fully node-local fashion. - -*Impact* + -Discontinue use of the `?local` query parameter. {ref}/cat-shards.html[cat shards -API] requests that include this parameter will return an error. -==== - -.The cat indices API's `local` query parameter has been removed. -[%collapsible] -==== -*Details* + -The `?local` parameter to the `GET _cat/indices` API was deprecated in 7.x and is -rejected in 8.0. This parameter caused the API to use the local cluster state -to determine the nodes returned by the API rather than the cluster state from -the master, but this API requests information from each selected node -regardless of the `?local` parameter which means this API does not run in a -fully node-local fashion. - -*Impact* + -Discontinue use of the `?local` query parameter. {ref}/cat-indices.html[cat indices -API] requests that include this parameter will return an error. -==== - -.The get field mapping API's `local` query parameter has been removed. -[%collapsible] -==== -*Details* + -The `local` parameter for get field mapping API was deprecated in 7.8 and is -removed in 8.0. This parameter is a no-op and field mappings are always retrieved -locally. - -*Impact* + -Discontinue use of the `local` query parameter. -{ref}/indices-get-field-mapping.html[get field mapping API] requests that -include this parameter will return an error. -==== - -.Post data to jobs API is deprecated. -[%collapsible] -==== -*Details* + -The {ml} {ref}/ml-post-data.html[post data to jobs API] is deprecated starting in 7.11.0 -and will be removed in a future major version. - -*Impact* + -Use {ref}/ml-apis.html#ml-api-datafeed-endpoint[{dfeeds}] instead. -==== - -.The `job_id` property of the Update {dfeeds} API has been removed. -[%collapsible] -==== -*Details* + -The ability to update a `job_id` in a {dfeed} was deprecated in 7.3.0. and is -removed in 8.0. - -*Impact* + -It is not possible to move {dfeeds} between {anomaly-jobs}. -==== - -.Create repository and delete repository API's return `409` status code when a repository is in use instead of `500`. -[%collapsible] -==== -*Details* + -The {ref}/put-snapshot-repo-api.html[Create or update snapshot repository API] and -{ref}/delete-snapshot-repo-api.html[Delete snapshot repository API] return `409` -status code when the request is attempting to modify an existing repository that's in use instead of status code `500`. - -*Impact* + -Update client code that handles creation and deletion of repositories to reflect this change. -==== - -.The `allow_no_datafeeds` property has been removed from {ml} APIs. -[%collapsible] -==== -*Details* + -The `allow_no_datafeeds` property was deprecated in the -{ref}/cat-datafeeds.html[cat {dfeeds}], -{ref}/ml-get-datafeed.html[get {dfeeds}], -{ref}/ml-get-datafeed-stats.html[get {dfeed} statistics], and -{ref}/ml-stop-datafeed.html[stop {dfeeds}] APIs in 7.10.0. - -*Impact* + -Use `allow_no_match` instead. -==== - -.The `allow_no_jobs` property has been removed from {ml} APIs. -[%collapsible] -==== -*Details* + -The `allow_no_jobs` property was deprecated in the -{ref}/cat-anomaly-detectors.html[cat anomaly detectors], -{ref}/ml-close-job.html[close {anomaly-jobs}], -{ref}/ml-get-job.html[get {anomaly-jobs}], -{ref}/ml-get-job-stats.html[get {anomaly-job} statistics], and -{ref}/ml-get-overall-buckets.html[get overall buckets] APIs in 7.10.0. - -*Impact* + -Use `allow_no_match` instead. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/breaker.asciidoc b/docs/reference/migration/migrate_8_0/breaker.asciidoc index d3c1be8d0dac2..c9b75d6b6f281 100644 --- a/docs/reference/migration/migrate_8_0/breaker.asciidoc +++ b/docs/reference/migration/migrate_8_0/breaker.asciidoc @@ -4,16 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `in_flight_requests` stat has been renamed `inflight_requests` in logs and diagnostic APIs. -[%collapsible] -==== -*Details* + -The name of the in flight requests circuit breaker in log output and diagnostic APIs (such as the node stats API) changes from `in_flight_requests` to `inflight_requests` to align it with the name of the corresponding settings. - -*Impact* + -Update your workflow and applications to use the `inflight_requests` stat in -place of `in_flight_requests`. -==== //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/ccr.asciidoc b/docs/reference/migration/migrate_8_0/ccr.asciidoc index f0c50afc94560..1f46d0b117d4f 100644 --- a/docs/reference/migration/migrate_8_0/ccr.asciidoc +++ b/docs/reference/migration/migrate_8_0/ccr.asciidoc @@ -4,17 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.Remote system indices are not followed automatically if they match an auto-follow pattern. -[%collapsible] -==== -*Details* + -Remote system indices matching an {ref}/ccr-auto-follow.html[auto-follow -pattern] won't be configured as a follower index automatically. - -*Impact* + -Explicitly {ref}/ccr-put-follow.html[create a follower index] to follow a remote -system index if that's the wanted behaviour. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/cluster.asciidoc b/docs/reference/migration/migrate_8_0/cluster.asciidoc index 950057a74c185..5a2adc45092ad 100644 --- a/docs/reference/migration/migrate_8_0/cluster.asciidoc +++ b/docs/reference/migration/migrate_8_0/cluster.asciidoc @@ -4,49 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The voting configuration exclusions API endpoint has changed. -[%collapsible] -==== -*Details* + -The `POST /_cluster/voting_config_exclusions/{node_filter}` API has been -removed in favour of `POST /_cluster/voting_config_exclusions?node_names=...` -and `POST /_cluster/voting_config_exclusions?node_ids=...` which allow you to -specify the names or IDs of the nodes to exclude. - -*Impact* + -Use `POST /_cluster/voting_config_exclusions?node_ids=...` and specify the nodes -to exclude instead of using a node filter. Requests submitted to the -`/_cluster/voting_config_exclusions/{node_filter}` endpoint will return an -error. -==== - -.The `cluster.join.timeout` setting has been removed. -[%collapsible] -==== -*Details* + -The `cluster.join.timeout` setting has been removed. Join attempts no longer -time out. - -*Impact* + -Do not set `cluster.join.timeout` in your `elasticsearch.yml` file. -==== - -.HTTP Status code has changed for the Cluster Health API in case of a server timeout. -[%collapsible] -==== -*Details* + -The {ref}/cluster-health.html[cluster health API] includes options for waiting -for certain health conditions to be satisfied. If the requested conditions are -not satisfied within a timeout then {es} will send back a normal response -including the field `"timed_out": true`. In earlier versions it would also use -the HTTP response code `408 Request timeout` if the request timed out, and `200 -OK` otherwise. The `408 Request timeout` response code is not appropriate for -this situation, so from version 8.0.0 {es} will use the response code `200 OK` -for both cases. - -*Impact* + -To detect a server timeout, check the `timed_out` field of the JSON response. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/discovery.asciidoc b/docs/reference/migration/migrate_8_0/discovery.asciidoc index 447e819b8bb01..f49f649b27072 100644 --- a/docs/reference/migration/migrate_8_0/discovery.asciidoc +++ b/docs/reference/migration/migrate_8_0/discovery.asciidoc @@ -4,42 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.`discovery.zen` settings have been removed. -[%collapsible] -==== -*Details* + -All settings under the `discovery.zen` namespace, which existed only for BWC reasons in 7.x, -are no longer supported. In particular, this includes: - -- `discovery.zen.minimum_master_nodes` -- `discovery.zen.no_master_block` -- `discovery.zen.hosts_provider` -- `discovery.zen.publish_timeout` -- `discovery.zen.commit_timeout` -- `discovery.zen.publish_diff.enable` -- `discovery.zen.ping.unicast.concurrent_connects` -- `discovery.zen.ping.unicast.hosts.resolve_timeout` -- `discovery.zen.ping.unicast.hosts` -- `discovery.zen.ping_timeout` -- `discovery.zen.unsafe_rolling_upgrades_enabled` -- `discovery.zen.fd.connect_on_network_disconnect` -- `discovery.zen.fd.ping_interval` -- `discovery.zen.fd.ping_timeout` -- `discovery.zen.fd.ping_retries` -- `discovery.zen.fd.register_connection_listener` -- `discovery.zen.join_retry_attempts` -- `discovery.zen.join_retry_delay` -- `discovery.zen.join_timeout` -- `discovery.zen.max_pings_from_another_master` -- `discovery.zen.send_leave_request` -- `discovery.zen.master_election.wait_for_joins_timeout` -- `discovery.zen.master_election.ignore_non_master_pings` -- `discovery.zen.publish.max_pending_cluster_states` - -*Impact* + -Discontinue use of the `discovery.zen` settings. Specifying these settings in -`elasticsearch.yml` will result in an error on startup. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/eql.asciidoc b/docs/reference/migration/migrate_8_0/eql.asciidoc index ffd58c960032b..6ad90cac1a247 100644 --- a/docs/reference/migration/migrate_8_0/eql.asciidoc +++ b/docs/reference/migration/migrate_8_0/eql.asciidoc @@ -4,15 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `wildcard` function has been removed. -[%collapsible] -==== -*Details* + -The `wildcard` function was deprecated in {es} 7.13.0 and has been removed. - -*Impact* + -Use the `like` or `regex` {ref}/eql-syntax.html#eql-syntax-pattern-comparison-keywords[keywords] instead. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/http.asciidoc b/docs/reference/migration/migrate_8_0/http.asciidoc index b22ee80135ccf..7db5b7ebbf0da 100644 --- a/docs/reference/migration/migrate_8_0/http.asciidoc +++ b/docs/reference/migration/migrate_8_0/http.asciidoc @@ -4,45 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `http.content_type.required` setting has been removed. -[%collapsible] -==== -*Details* + -The `http.content_type.required` setting was deprecated in Elasticsearch 6.0 -and has been removed in Elasticsearch 8.0. The setting was introduced in -Elasticsearch 5.3 to prepare users for Elasticsearch 6.0, where content type -auto detection was removed for HTTP requests. - -*Impact* + -Discontinue use of the `http.content_type.required` setting. Specifying this -setting in `elasticsearch.yml` will result in an error on startup. -==== - -.The `http.tcp_no_delay` setting has been replaced by `http.tcp.no_delay`. -[%collapsible] -==== -*Details* + -The `http.tcp_no_delay` setting was deprecated in 7.x and has been removed in 8.0. It has been replaced by -`http.tcp.no_delay`. - -*Impact* + -Use the `http.tcp.no_delay` setting. Discontinue use of the `http.tcp_no_delay` -setting. Specifying the `http.tcp_no_delay` setting in `elasticsearch.yml` will -result in an error on startup. -==== - -.The `es.rest.url_plus_as_space` system property has been removed. -[%collapsible] -==== -*Details* + -Starting in version 7.4, a `+` in a URL will be encoded as `%2B` by all REST API functionality. Prior versions handled a `+` as a single space. -In these previous versions, if your application required handling `+` as a single space, you could return to the old behaviour by setting the system property -`es.rest.url_plus_as_space` to `true`. Note that this behaviour is deprecated and setting this system property to `true` will cease -to be supported in version 8. - -*Impact* + -Update your application or workflow to assume `+` in a URL is encoded as `%2B`. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/ilm.asciidoc b/docs/reference/migration/migrate_8_0/ilm.asciidoc index fa5042c33a0fc..0f068006991f0 100644 --- a/docs/reference/migration/migrate_8_0/ilm.asciidoc +++ b/docs/reference/migration/migrate_8_0/ilm.asciidoc @@ -4,64 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[ilm-poll-interval-limit]] -.The `indices.lifecycle.poll_interval` setting must be greater than `1s`. -[%collapsible] -==== -*Details* + -The setting `indices.lifecycle.poll_interval`, if set too low, can cause -excessive load on a cluster. This setting must now be set to `1s` (one second) -or greater. - -*Impact* + -Update the `indices.lifecycle.poll_interval` setting to a value of `1s` or -greater using `elasticsearch.yml` or the -{ref}/cluster-update-settings.html[cluster update settings API]. - -Setting `indices.lifecycle.poll_interval` to less than `1s` in -`elasticsearch.yml` will result in an error on startup. -{ref}/cluster-update-settings.html[Cluster update settings API] requests that -set `indices.lifecycle.poll_interval` to less than `1s` will return an error. -==== - -[[ilm-hlrc-rename]] -.The `indexlifecycle` package has been renamed `ilm` in the Java High Level REST Client. -[%collapsible] -==== -*Details* + -In the high level REST client, the `indexlifecycle` package has been -renamed to `ilm` to match the package rename inside the {es} code. - -*Impact* + -Update your workflow and applications to use the `ilm` package in place of -`indexlifecycle`. -==== - -[[ilm-freeze-noop]] -.The ILM `freeze` action is now a no-op. -[%collapsible] -==== -*Details* + -The ILM freeze action is now a no-op and performs no action on the index, as the freeze API endpoint -has been removed in 8.0. - -*Impact* + -Update your ILM policies to remove the `freeze` action from the `cold` phase. -==== - -[[ilm-policy-validation]] -.Additional validation for ILM policies. -[%collapsible] -==== -*Details* + -Creating or updating an ILM policy now requires that any referenced snapshot repositories and SLM -policies exist. - -*Impact* + -Update your code or configuration management to ensure that repositories and SLM policies are created -before any policies that reference them. -==== - -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/indices.asciidoc b/docs/reference/migration/migrate_8_0/indices.asciidoc index 585233c736708..9277300fe04bf 100644 --- a/docs/reference/migration/migrate_8_0/indices.asciidoc +++ b/docs/reference/migration/migrate_8_0/indices.asciidoc @@ -4,145 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The deprecated `_upgrade` API has been removed. -[%collapsible] -==== -*Details* + -Previously, the `_upgrade` API upgraded indices from the previous major -version to the current version. The `_reindex` API should be used -instead for that purpose. - -*Impact* + -Requests made to the old `_upgrade` API will return an error. -==== - -.The deprecated freeze index API has been removed. -[%collapsible] -==== -*Details* + -The freeze index API (`POST //_freeze`) has been removed. -https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage[Improvements -in heap memory usage] have eliminated the reason to freeze indices. -You can still unfreeze existing frozen indices using the -{ref}/unfreeze-index-api.html[unfreeze index API]. For some use cases, the -frozen tier may be a suitable replacement for frozen indices. See -{ref}/data-tiers.html[data tiers] for more information. - -*Impact* + -Requests made to the old freeze index API will return an error. -==== - -.The force merge API's `max_num_segments` and `only_expunge_deletes` parameters cannot both be specified in the same request. -[%collapsible] -==== -*Details* + -Previously, the force merge API allowed the parameters `only_expunge_deletes` -and `max_num_segments` to be set to a non default value at the same time. But -the `max_num_segments` was silently ignored when `only_expunge_deletes` is set -to `true`, leaving the false impression that it has been applied. - -*Impact* + -When using the {ref}/indices-forcemerge.html[force merge API], do not specify -values for both the `max_num_segments` and `only_expunge_deletes` parameters. -Requests that include values for both parameters will return an error. -==== - -.The `index.force_memory_term_dictionary` setting has been removed. -[%collapsible] -==== -*Details* + -The `index.force_memory_term_dictionary` setting was introduced in 7.0 as a -temporary measure to allow users to opt-out of the optimization that leaves the -term dictionary on disk when appropriate. This optimization is now mandatory -and the setting is removed. - -*Impact* + -Discontinue use of the `index.force_memory_term_dictionary` index setting. -Requests that include this setting will return an error. -==== - -.The create or update index template API's `template` parameter has been removed. -[%collapsible] -==== -*Details* + -In 6.0, we deprecated the `template` parameter in create or update index -template requests in favor of using `index_patterns`. Support for the `template` -parameter is now removed in 8.0. - -*Impact* + -Use the {ref}/indices-templates-v1.html[create or update index template API]'s -`index_patterns` parameter. Requests that include the `template` parameter will -return an error. -==== - -.Synced flush has been removed. -[%collapsible] -==== -*Details* + -Synced flush was deprecated in 7.6 and is removed in 8.0. Use a regular flush -instead as it has the same effect as a synced flush in 7.6 and later. - -*Impact* + -Use the {ref}/indices-flush.html[flush API]. Requests to the -`//flush/synced` or `/flush/synced` endpoints will return an error. -==== - -.The `index.soft_deletes.enabled` setting has been removed. -[%collapsible] -==== -*Details* + -Creating indices with soft deletes disabled was deprecated in 7.6 and -is no longer supported in 8.0. The `index.soft_deletes.enabled` setting -can no longer be set to `false`. - -*Impact* + -Discontinue use of the `index.soft_deletes.enabled` index setting. Requests that -set `index.soft_deletes.enabled` to `false` will return an error. -==== - -.The `index.translog.retention.age` and `index.translog.retention.size` settings have been removed. -[%collapsible] -==== -*Details* + -Translog retention settings `index.translog.retention.age` and -`index.translog.retention.size` were effectively ignored in 7.4, deprecated in -7.7, and removed in 8.0 in favor of -{ref}/index-modules-history-retention.html[soft deletes]. - -*Impact* + -Discontinue use of the `index.translog.retention.age` and -`index.translog.retention.size` index settings. Requests that -include these settings will return an error. -==== - -.The default for the `?wait_for_active_shards` parameter on the close index API has changed. -[%collapsible] -==== -*Details* + -When closing an index in earlier versions, by default {es} would not wait for -the shards of the closed index to be properly assigned before returning. From -version 8.0 onwards the default behaviour is to wait for shards to be assigned -according to the -{ref}/docs-index_.html#index-wait-for-active-shards[`index.write.wait_for_active_shards` -index setting]. - -*Impact* + -Accept the new behaviour, or specify `?wait_for_active_shards=0` to preserve -the old behaviour if needed. -==== - -.The index stats API's `types` query parameter has been removed. -[%collapsible] -==== -*Details* + -The index stats API's `types` query parameter has been removed. Previously, you -could combine `types` with the `indexing` query parameter to return indexing -stats for specific mapping types. Mapping types have been removed in 8.0. - -*Impact* + -Discontinue use of the `types` query parameter. Requests that include the -parameter will return an error. -==== //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/ingest.asciidoc b/docs/reference/migration/migrate_8_0/ingest.asciidoc index b9308988b870d..9cddd826ad115 100644 --- a/docs/reference/migration/migrate_8_0/ingest.asciidoc +++ b/docs/reference/migration/migrate_8_0/ingest.asciidoc @@ -4,46 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `user_agent` ingest processor's `ecs` parameter has no effect. -[%collapsible] -==== -*Details* + -In 7.2, we deprecated the `ecs` parameter for the `user_agent` ingest processor. -In 8.x, the `user_agent` ingest processor will only return {ecs-ref}[Elastic -Common Schema (ECS)] fields, regardless of the `ecs` value. - -*Impact* + -To avoid deprecation warnings, remove the parameter from your ingest pipelines. -If a pipeline specifies an `ecs` value, the value is ignored. -==== - -.The default Maxmind geoip databases have been removed. -[%collapsible] -==== -*Details* + -The default Maxmind geoip databases that shipped by default with Elasticsearch -have been removed. These databases are out dated and stale and using these -databases will likely result in incorrect geoip lookups. - -By default since 7.13, these pre-packaged geoip databases -were used in case no database were specified in the config directory or before -the geoip downloader downloaded the geoip databases. When the geoip database -downloader completed downloading the new databases then these pre-packaged -databases were no longer used. - -*Impact* + -If the geoip downloader is disabled and no geoip databases are provided -in the config directory of each ingest node then the geoip processor will -no longer perform geoip lookups and tag these documents with the fact that -the requested database is no longer available. - -After a cluster has been started and before the geoip downloader has completed -downloading the most up to data databases, the geoip processor will not perform -any geoip lookups and tag documents that the requested database is not available. -After the geoip downloader has completed downloading the most up to data databases -then the geoip processor will function as normal. The window of time that the -geoip processor can't do geoip lookups after cluster startup should be very small. -==== //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/java.asciidoc b/docs/reference/migration/migrate_8_0/java.asciidoc index ec5b9f7617542..4a0439cd58993 100644 --- a/docs/reference/migration/migrate_8_0/java.asciidoc +++ b/docs/reference/migration/migrate_8_0/java.asciidoc @@ -4,39 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.Changes to `Fuzziness`. -[%collapsible] -==== -*Details* + -To create `Fuzziness` instances, use the `fromString` and `fromEdits` method -instead of the `build` method that used to accept both Strings and numeric -values. Several fuzziness setters on query builders (e.g. -MatchQueryBuilder#fuzziness) now accept only a `Fuzziness`instance instead of -an Object. - -Fuzziness used to be lenient when it comes to parsing arbitrary numeric values -while silently truncating them to one of the three allowed edit distances 0, 1 -or 2. This leniency is now removed and the class will throw errors when trying -to construct an instance with another value (e.g. floats like 1.3 used to get -accepted but truncated to 1). - -*Impact* + -Use the available constants (e.g. `Fuzziness.ONE`, `Fuzziness.AUTO`) or build -your own instance using the above mentioned factory methods. Use only allowed -`Fuzziness` values. -==== - -.Changes to `Repository`. -[%collapsible] -==== -*Details* + -Repository has no dependency on IndexShard anymore. The contract of restoreShard -and snapshotShard has been reduced to Store and MappingService in order to improve -testability. - -*Impact* + -No action needed. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/logging.asciidoc b/docs/reference/migration/migrate_8_0/logging.asciidoc index 90f2c104d9325..2245b2f08fbc3 100644 --- a/docs/reference/migration/migrate_8_0/logging.asciidoc +++ b/docs/reference/migration/migrate_8_0/logging.asciidoc @@ -4,39 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.{es} JSON logs now comply with ECS. -[%collapsible] -==== -*Details* + -{es}'s {ref}/logging.html[JSON logs] now comply with the -{ecs-ref}/index.html[Elastic Common Schema (ECS)]. Previously, {es}'s JSON logs -used a custom schema. - -*Impact* + -If your application parses {es}'s JSON logs, update it to support the new ECS -format. -==== - - -.{es} no longer emits deprecation logs or slow logs in plaintext. -[%collapsible] -==== -*Details* + -{es} no longer emits a plaintext version of the following logs: - -* Deprecation logs -* Indexing slow logs -* Search slow logs - -These logs are now only available in JSON. - -Server logs are still available in both a JSON and plaintext format. - -*Impact* + -If your application parses {es}'s plaintext logs, update it to use the new ECS -JSON logs. -==== - -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/mappings.asciidoc b/docs/reference/migration/migrate_8_0/mappings.asciidoc index 71721120dd7d7..3dbeb7893581a 100644 --- a/docs/reference/migration/migrate_8_0/mappings.asciidoc +++ b/docs/reference/migration/migrate_8_0/mappings.asciidoc @@ -4,121 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The maximum number of completion contexts per field is now 10. -[%collapsible] -==== -*Details* + -The number of completion contexts within a single completion field -has been limited to 10. - -*Impact* + -Use a maximum of 10 completion contexts in a completion field. Specifying more -than 10 completion contexts will return an error. -==== - -.Mapping API endpoints containing mapping types have been removed. -[%collapsible] -==== -*Details* + -The typed REST endpoints of the update mapping, get mapping and get field mapping -APIs have been removed in favour of their typeless REST endpoints, since indexes -no longer contain types, these typed endpoints are obsolete. - -*Impact* + -Use the typeless REST endpoints to update and retrieve mappings. Requests -submitted to the typed mapping API endpoints will return an error. -==== - -.Multi-fields within multi-fields is no longer supported. -[%collapsible] -==== -*Details* + -Previously, it was possible to define a multi-field within a multi-field. -Defining chained multi-fields was deprecated in 7.3 and is now no longer -supported. - -*Impact* + -To migrate mappings, all instances of `fields` that occur within -a `fields` block should be removed, either by flattening the chained `fields` -blocks into a single level, or by switching to `copy_to` if appropriate. -==== - -[[fieldnames-enabling]] -.The `_field_names` metadata field's `enabled` parameter has been removed. -[%collapsible] -==== -*Details* + -The setting has been deprecated with 7.5 and is no longer supported on new indices. -Mappings for older indices will continue to work but emit a deprecation warning. - -*Impact* + -The `enabled` setting for `_field_names` should be removed from templates and mappings. -Disabling _field_names is not necessary because it no longer carries a large index overhead. -==== - -[[mapping-boosts]] -.The `boost` parameter on field mappings has been removed. -[%collapsible] -==== -*Details* + -Index-time boosts have been deprecated since the 5x line, but it was still possible -to declare field-specific boosts in the mappings. This is now removed completely. -Indexes built in 7x that contain mapping boosts will emit warnings, and the boosts -will have no effect in 8.0. New indexes will not permit boosts to be set in their -mappings at all. - -*Impact* + -The `boost` setting should be removed from templates and mappings. Use boosts -directly on queries instead. -==== - -.Java-time date formats replace joda-time formats. -[%collapsible] -==== -*Details* + -In 7.0, {es} switched from joda time to java time for date-related parsing, -formatting, and calculations. Indices created in 7.0 and later versions are -already required to use mappings with java-time date formats. However, -earlier indices using joda-time formats must be reindexed to use -mappings with java-time formats. - -*Impact* + -For a detailed migration guide, see the {ref}/migrate-to-java-time.html[Java -time migration guide]. -==== - -[[geo-shape-strategy]] -.Several `geo_shape` mapping parameters have been removed. -[%collapsible] -==== -*Details* + -The following `geo_shape` mapping parameters were deprecated in 6.6: - -* `tree` -* `tree_levels` -* `strategy` -* `distance_error_pct` - -These parameters have been removed in 8.0.0. - -*Impact* + -In 8.0, you can no longer create mappings that include these parameters. -However, 7.x indices that use these mapping parameters will continue to work. -==== - -.The `include_type_name` query parameter has been removed. -[%collapsible] -==== -*Details* + -The `include_type_name` query parameter has been removed from the index -creation, index template, and mapping APIs. Previously, you could set -`include_type_name` to `true` to indicate that requests and responses should -include a mapping type name. Mapping types have been removed in 8.x. - -*Impact* + -Discontinue use of the `include_type_name` query parameter. Requests that -include the parameter will return an error. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/network.asciidoc b/docs/reference/migration/migrate_8_0/network.asciidoc index 158ee70f70a2e..abb1592deb9f4 100644 --- a/docs/reference/migration/migrate_8_0/network.asciidoc +++ b/docs/reference/migration/migrate_8_0/network.asciidoc @@ -4,20 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `network.tcp.connect_timeout` setting has been removed. -[%collapsible] -==== -*Details* + -The `network.tcp.connect_timeout` setting was deprecated in 7.x and has been removed in 8.0. This setting -was a fallback setting for `transport.connect_timeout`. - -*Impact* + -Use the `transport.connect_timeout` setting to change the default connection -timeout for client connections. Discontinue use of the -`network.tcp.connect_timeout` setting. Specifying the -`network.tcp.connect_timeout` setting in `elasticsearch.yml` will result in an -error on startup. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/node.asciidoc b/docs/reference/migration/migrate_8_0/node.asciidoc index f5b09bffa440c..4a5dbda5d52c5 100644 --- a/docs/reference/migration/migrate_8_0/node.asciidoc +++ b/docs/reference/migration/migrate_8_0/node.asciidoc @@ -4,59 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `node.max_local_storage_nodes` setting has been removed. -[%collapsible] -==== -*Details* + -The `node.max_local_storage_nodes` setting was deprecated in 7.x and -has been removed in 8.0. Nodes should be run on separate data paths -to ensure that each node is consistently assigned to the same data path. - -*Impact* + -Discontinue use of the `node.max_local_storage_nodes` setting. Specifying this -setting in `elasticsearch.yml` will result in an error on startup. -==== - -.The layout of the data folder has changed. -[%collapsible] -==== -*Details* + -Each node's data is now stored directly in the data directory set by the -`path.data` setting, rather than in `${path.data}/nodes/0`, because the removal -of the `node.max_local_storage_nodes` setting means that nodes may no longer -share a data path. - -*Impact* + -At startup, {es} will automatically migrate the data path to the new layout. -This automatic migration will not proceed if the data path contains data for -more than one node. You should move to a configuration in which each node has -its own data path before upgrading. - -If you try to upgrade a configuration in which there is data for more than one -node in a data path then the automatic migration will fail and {es} -will refuse to start. To resolve this you will need to perform the migration -manually. The data for the extra nodes are stored in folders named -`${path.data}/nodes/1`, `${path.data}/nodes/2` and so on, and you should move -each of these folders to an appropriate location and then configure the -corresponding node to use this location for its data path. If your nodes each -have more than one data path in their `path.data` settings then you should move -all the corresponding subfolders in parallel. Each node uses the same subfolder -(e.g. `nodes/2`) across all its data paths. -==== - -.Closed indices created in {es} 6.x and earlier versions are not supported. -[%collapsible] -==== -*Details* + -In earlier versions a node would start up even if it had data from indices -created in a version before the previous major version, as long as those -indices were closed. {es} now ensures that it is compatible with every index, -open or closed, at startup time. - -*Impact* + -Reindex closed indices created in {es} 6.x or before with {es} 7.x if they need -to be carried forward to {es} 8.x. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/packaging.asciidoc b/docs/reference/migration/migrate_8_0/packaging.asciidoc index 2962b783cc3e2..7cf570ea92498 100644 --- a/docs/reference/migration/migrate_8_0/packaging.asciidoc +++ b/docs/reference/migration/migrate_8_0/packaging.asciidoc @@ -4,35 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.Java 17 is required. -[%collapsible] -==== -*Details* + -Java 17 or higher is now required to run {es} and any of its command -line tools. - -*Impact* + -Use Java 17 or higher. Attempts to run {es} 8.0 using earlier Java versions will -fail. - -There is not yet a FIPS-certified security module for Java 17 -that you can use when running {es} 8.0 in FIPS 140-2 mode. -If you run in FIPS 140-2 mode, you will either need to request an exception -from your security organization to upgrade to {es} 8.0, -or remain on {es} 7.x until Java 17 is certified. -==== - -.JAVA_HOME is no longer supported. -[%collapsible] -==== -*Details* + -`JAVA_HOME` is no longer supported to set the path for the JDK. Instead, use -the bundled JDK (preferable), or set `ES_JAVA_HOME`. - -*Impact* + -Use the bundled JDK (preferable), or set `ES_JAVA_HOME`. `JAVA_HOME` will be -ignored. -==== //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/reindex.asciidoc b/docs/reference/migration/migrate_8_0/reindex.asciidoc index 875229c57ecd8..25f65edc18115 100644 --- a/docs/reference/migration/migrate_8_0/reindex.asciidoc +++ b/docs/reference/migration/migrate_8_0/reindex.asciidoc @@ -4,63 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.Reindex from remote now re-encodes URL-encoded index names. -[%collapsible] -==== -*Details* + -Reindex from remote would previously allow URL-encoded index names and not -re-encode them when generating the search request for the remote host. This -leniency has been removed such that all index names are correctly encoded when -reindex generates remote search requests. - -*Impact* + -Specify unencoded index names for reindex from remote requests. -==== - -.Reindex-related REST API endpoints containing mapping types have been removed. -[%collapsible] -==== -*Details* + -The `/{index}/{type}/_delete_by_query` and `/{index}/{type}/_update_by_query` REST endpoints have been removed in favour of `/{index}/_delete_by_query` and `/{index}/_update_by_query`, since indexes no longer contain types, these typed endpoints are obsolete. - -*Impact* + -Use the replacement REST API endpoints. Requests submitted to API endpoints -that contain a mapping type will return an error. -==== - - -.In the reindex, delete by query, and update by query APIs, the `size` parameter has been renamed. -[%collapsible] -==== -*Details* + -Previously, a `_reindex` request had two different size specifications in the body: - -- Outer level, determining the maximum number of documents to process -- Inside the `source` element, determining the scroll/batch size. - -The outer level `size` parameter has now been renamed to `max_docs` to -avoid confusion and clarify its semantics. - -Similarly, the `size` parameter has been renamed to `max_docs` for -`_delete_by_query` and `_update_by_query` to keep the 3 interfaces consistent. - -*Impact* + -Use the replacement parameters. Requests containing the `size` parameter will -return an error. -==== - -.The update by query API now rejects unsupported `script` fields. -[%collapsible] -==== -*Details* + -An update by query API request that includes an unsupported field in the -`script` object now returns an error. Previously, the API would silently ignore -these unsupported fields. - -*Impact* + -To avoid errors, remove unsupported fields from the `script` object. -==== - //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/rollup.asciidoc b/docs/reference/migration/migrate_8_0/rollup.asciidoc index 4cf1d448fe233..e49aeb50b6c13 100644 --- a/docs/reference/migration/migrate_8_0/rollup.asciidoc +++ b/docs/reference/migration/migrate_8_0/rollup.asciidoc @@ -4,23 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The StartRollupJob endpoint now returns a success status if a job has already started. -[%collapsible] -==== -*Details* + -Previously, attempting to start an already-started rollup job would -result in a `500 InternalServerError: Cannot start task for Rollup Job -[job] because state was [STARTED]` exception. - -Now, attempting to start a job that is already started will just -return a successful `200 OK: started` response. - -*Impact* + -Update your workflow and applications to assume that a 200 status in response to -attempting to start a rollup job means the job is in an actively started state. -The request itself may have started the job, or it was previously running and so -the request had no effect. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/scripting.asciidoc b/docs/reference/migration/migrate_8_0/scripting.asciidoc index 05141efc37435..0619714292035 100644 --- a/docs/reference/migration/migrate_8_0/scripting.asciidoc +++ b/docs/reference/migration/migrate_8_0/scripting.asciidoc @@ -4,70 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `JodaCompatibleZonedDateTime` class has been removed. -[%collapsible] -==== -*Details* + -As a transition from Joda datetime to Java datetime, scripting used -an intermediate class called `JodaCompatibleZonedDateTime`. This class -has been removed and is replaced by `ZonedDateTime`. Any use of casting -to a `JodaCompatibleZonedDateTime` or use of method calls only available -in `JodaCompatibleZonedDateTime` in a script will result in a compilation -error, and may not allow the upgraded node to start. - -*Impact* + -Before upgrading, replace `getDayOfWeek` with `getDayOfWeekEnum().value` in any -scripts. Any use of `getDayOfWeek` expecting a return value of `int` will result -in a compilation error or runtime error and may not allow the upgraded node to -start. - -The following `JodaCompatibleZonedDateTime` methods must be replaced using -`ZonedDateTime` methods prior to upgrade: - -* `getMillis()` -> `toInstant().toEpochMilli()` -* `getCenturyOfEra()` -> `get(ChronoField.YEAR_OF_ERA) / 100` -* `getEra()` -> `get(ChronoField.ERA)` -* `getHourOfDay()` -> `getHour()` -* `getMillisOfDay()` -> `get(ChronoField.MILLI_OF_DAY)` -* `getMillisOfSecond()` -> `get(ChronoField.MILLI_OF_SECOND)` -* `getMinuteOfDay()` -> `get(ChronoField.MINUTE_OF_DAY)` -* `getMinuteOfHour()` -> `getMinute()` -* `getMonthOfYear()` -> `getMonthValue()` -* `getSecondOfDay()` -> `get(ChronoField.SECOND_OF_DAY)` -* `getSecondOfMinute()` -> `getSecond()` -* `getWeekOfWeekyear()` -> `get(DateFormatters.WEEK_FIELDS_ROOT.weekBasedYear())` -* `getYearOfCentury()` -> `get(ChronoField.YEAR_OF_ERA) % 100` -* `getYearOfEra()` -> `get(ChronoField.YEAR_OF_ERA)` -* `toString(String)` -> a DateTimeFormatter -* `toString(String, Locale)` -> a DateTimeFormatter -==== - -.Stored scripts no longer support empty scripts or search templates. -[%collapsible] -==== -*Details* + -The {ref}/create-stored-script-api.html[create or update stored script API]'s -`source` parameter cannot be empty. - -*Impact* + -Before upgrading, use the {ref}/delete-stored-script-api.html[delete stored -script API] to delete any empty stored scripts or search templates. -In 8.0, {es} will drop any empty stored scripts or empty search templates from -the cluster state. Requests to create a stored script or search template with -an empty `source` will return an error. -==== - -.The create or update stored script API's `code` parameter has been removed. -[%collapsible] -==== -*Details* + -The {ref}/create-stored-script-api.html[create or update stored script API]'s -`code` parameter has been removed. Use the `source` parameter instead. - -*Impact* + -Discontinue use of the `code` parameter. Requests that include the parameter -will return an error. -==== -// end::notable-breaking-changes[] \ No newline at end of file +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/search.asciidoc b/docs/reference/migration/migrate_8_0/search.asciidoc index 6fe9b400e0489..4dd866110a34b 100644 --- a/docs/reference/migration/migrate_8_0/search.asciidoc +++ b/docs/reference/migration/migrate_8_0/search.asciidoc @@ -4,262 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[_type-search-matches-no-docs]] -.Searches on the `_type` field are no longer supported. -[%collapsible] -==== -*Details* + -In 8.x, the `_type` metadata field has been removed. {es} now handles a search -on the `_type` field as a search on a non-existent field. A search on a -non-existent field matches no documents, regardless of the query string. - -In 7.x, a search for `_doc` in the `_type` field would match the same documents -as a `match_all` query. - -*Impact* + -Remove queries on the `_type` field from your search requests and search -templates. Searches that include these queries may return no results. -==== - -[[msearch-empty-line-support]] -.The multi search API now parses an empty first line as action metadata in text files. -[%collapsible] -==== -*Details* + -The multi search API now parses an empty first line as empty action metadata -when you provide a text file as the request body, such as when using curl's -`--data-binary` flag. - -The API no longer supports text files that contain: - -* An empty first line followed by a line containing only `{}`. -* An empty first line followed by another empty line. - -*Impact* + -Don't provide an unsupported text file to the multi search API. Requests that -include an unsupported file will return an error. -==== - -[[remove-unmapped-type-string]] -.The `unmapped_type: string` sort option has been removed. -[%collapsible] -==== -*Details* + -Search requests no longer support the `unmapped_type: string` sort option. -Instead, use `unmapped_type: keyword` to handle an unmapped field as if it had -the `keyword` field type but ignore its values for sorting. - -*Impact* + -Discontinue use of `unmapped_type: string`. Search requests that include the -`unmapped_type: string` sort option will return shard failures. -==== - -[[id-field-data]] -.Aggregating and sorting on `_id` is disallowed by default. -[%collapsible] -==== -*Details* + -Previously, it was possible to aggregate and sort on the built-in `_id` field -by loading an expensive data structure called fielddata. This was deprecated -in 7.6 and is now disallowed by default in 8.0. - -*Impact* + -Aggregating and sorting on `_id` should be avoided. As an alternative, the -`_id` field's contents can be duplicated into another field with docvalues -enabled (note that this does not apply to auto-generated IDs). -==== - -[[max_clause_count_change]] -.The `indices.query.bool.max_clause_count` setting now limits all query clauses. -[%collapsible] -==== -*Details* + -Previously, the `indices.query.bool.max_clause_count` would apply to the number -of clauses of a single `bool` query. It now applies to the total number of -clauses of the rewritten query. In order to reduce chances of breaks, its -default value has been bumped from 1024 to 4096. - -*Impact* + -Queries with many clauses should be avoided whenever possible. If you had bumped -this setting already in order to accomodate for some heavy queries, you might -need to bump it further so that these heavy queries keep working. -==== - -.Search-related REST API endpoints containing mapping types have been removed. -[%collapsible] -==== -*Details* + -The `/{index}/{type}/_search`, `/{index}/{type}/_msearch`, `/{index}/{type}/_search/template` and `/{index}/{type}/_msearch/template` REST endpoints have been removed in favour of `/{index}/_search`, `/{index}/_msearch`, `/{index}/_search/template` and `/{index}/_msearch/template`; since indexes no longer contain types, these typed endpoints are obsolete.. - -The `/{index}/{type}/_termvectors`, `/{index}/{type}/{id}/_termvectors` and `/{index}/{type}/_mtermvectors` REST endpoints have been removed in favour of `/{index}/_termvectors`, `/{index}/{id}/_termvectors` and `/{index}/_mtermvectors`; since indexes no longer contain types, these typed endpoints are obsolete.. - -The `/{index}/{type}/{doc}` and `/{index}/{type}/_mget` REST endpoints have been removed in favour of `/{index}/_doc/{doc}` and `/{index}/_mget`; since indexes no longer contain types, these typed endpoints are obsolete. - -*Impact* + -Use the replacement REST API endpoints. Requests submitted to API endpoints that -contain a mapping type will return an error. -==== - -.The `common` query has been removed. -[%collapsible] -==== -*Details* + -The `common` query, deprecated in 7.x, has been removed in 8.0. -The same functionality can be achieved by the `match` query if the total number of hits is not tracked. - -*Impact* + -Discontinue use of the `common` query. Search requests containing a `common` -query will return an error. -==== - -.The `cutoff_frequency` parameter has been removed from the `match` and `multi_match` query. -[%collapsible] -==== -*Details* + -The `cutoff_frequency` parameter, deprecated in 7.x, has been removed in 8.0 from `match` and `multi_match` queries. -The same functionality can be achieved without any configuration provided that the total number of hits is not tracked. - -*Impact* + -Discontinue use of the `cutoff_frequency` parameter. Search requests containing -this parameter in a `match` or `multi_match` query will return an error. -==== - -.The `nested_filter` and `nested_path` properties have been removed from the search API's `sort` request body parameter. -[%collapsible] -==== -*Details* + -The `nested_filter` and `nested_path` options, deprecated in 6.x, have been removed in favor of the `nested` context. - -*Impact* + -Discontinue use of the `sort` request body parameter's `nested_filter` and -`nested_path` properties. Requests containing these properties will return an -error. -==== - -.Search and get requests are now routed to shards using adaptive replica selection by default. -[%collapsible] -==== -*Details* + -{es} will no longer prefer using shards in the same location (with the same awareness attribute values) to process -`_search` and `_get` requests. Adaptive replica selection (activated by default in this version) will route requests -more efficiently using the service time of prior inter-node communications. - -*Impact* + -No action needed. -==== - -.The `sparse_vector` field data type has been removed. -[%collapsible] -==== -*Details* + -The `sparse_vector` field type was deprecated in 7.6 and is now removed in -8.0. We have not seen much interest in this experimental field type, and don't -see a clear use case as it's currently designed. If you have feedback or -suggestions around sparse vector functionality, please let us know through -GitHub or the 'discuss' forums. - -*Impact* + -Discontinue use of the `sparse_vector` field data type. Requests containing -a mapping for this field data type will return an error. -==== - -.Vector functions using `(query, doc['field'])` are no longer supported. -[%collapsible] -==== -*Details* + -The vector functions of the form `function(query, doc['field'])` were -deprecated in 7.6, and are now removed in 8.x. The form -`function(query, 'field')` should be used instead. For example, -`cosineSimilarity(query, doc['field'])` is replaced by -`cosineSimilarity(query, 'field')`. - -*Impact* + -Use the `function(query, 'field')` form. Discontinue use of the `function(query, -doc['field'])` form. Requests containing the `function(query, -doc['field'])` form will return an error. -==== - -.The search API's `indices_boost` request body parameter no longer accepts object values. -[%collapsible] -==== -*Details* + -The `indices_boost` option in the search request used to accept the boosts -both as an object and as an array. The object format has been deprecated since -5.2 and is now removed in 8.0. - -*Impact* + -Use only array values in the `indices_boost` parameter. Requests containing an -object value in the `indices_boost` parameter will return an error. -==== - -.The search API's `use_field_mapping` request body parameter has been removed. -[%collapsible] -==== -*Details* + -In 7.0, we began formatting `docvalue_fields` by default using each field's -mapping definition. To ease the transition from 6.x, we added the format -option `use_field_mapping`. This parameter was deprecated in 7.0, and is now -removed in 8.0. - -*Impact* + -Discontinue use of the `use_field_mapping` request body parameter. Requests -containing this parameter will return an error. -==== - - -.The search API's `from` request body and url parameter cannot be negative. -[%collapsible] -==== -*Details* + -Search request used to accept `-1` as a `from` in the search body and the url, -treating it as the default value of 0. Other negative values got rejected with -an error already. We now also reject `-1` as an invalid value. - -*Impact* + -Change any use of `-1` as `from` parameter in request body or url parameters by either -setting it to `0` or omitting it entirely. Requests containing negative values will -return an error. -==== - -.Range queries on date fields treat numeric values alwas as milliseconds-since-epoch. -[%collapsible] -==== -*Details* + -Range queries on date fields used to misinterpret small numbers (e.g. four digits like 1000) -as a year when no additional format was set, but would interpret other numeric values as -milliseconds since epoch. We now treat all numeric values in absence of a specific `format` -parameter as milliseconds since epoch. If you want to query for years instead, with a missing -`format` you now need to quote the input value (e.g. "1984"). - -*Impact* + -If you query date fields without a specified `format`, check if the values in your queries are -actually meant to be milliseconds-since-epoch and use a numeric value in this case. If not, use -a string value which gets parsed by either the date format set on the field in the mappings or -by `strict_date_optional_time` by default. -==== - -.The `geo_bounding_box` query's `type` parameter has been removed. -[%collapsible] -==== -*Details* + -The `geo_bounding_box` query's `type` parameter was deprecated in 7.14.0 and has -been removed in 8.0.0. This parameter is a no-op and has no effect on the query. - -*Impact* + -Discontinue use of the `type` parameter. `geo_bounding_box` queries that include -this parameter will return an error. -==== - -.The `type` query has been removed. -[%collapsible] -==== -*Details* + -The `type` query has been removed. Mapping types have been removed in 8.0. - -*Impact* + -Discontinue use of the `type` query. Requests that include the `type` query -will return an error. -==== //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/security.asciidoc b/docs/reference/migration/migrate_8_0/security.asciidoc index 1ff1580f6e264..7eb4bbf7813fb 100644 --- a/docs/reference/migration/migrate_8_0/security.asciidoc +++ b/docs/reference/migration/migrate_8_0/security.asciidoc @@ -4,422 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[deprecate-elasticsearch-setup-passwords]] -.The `elasticsearch-setup-passwords` tool is deprecated. -[%collapsible] -==== -*Details* + -In 8.0, we're deprecating the `elasticsearch-setup-passwords` tool. To -manually reset the password for built-in users (including the `elastic` user), use -the {ref}/reset-password.html[`elasticsearch-reset-password`] tool, the {es} -{ref}/security-api-change-password.html[change passwords API], or the -User Management features in {kib}. We will remove the -`elasticsearch-setup-passwords` tool in a future release. - -*Impact* + -When starting {es} for the first time, passwords are generated automatically for -the `elastic` user. If you run the `elasticsearch-setup-passwords` tool after -starting {es}, the command will fail because the password for the `elastic` -user is already configured. -==== - -.The file and native realms are now enabled unless explicitly disabled. -[%collapsible] -==== -*Details* + -The file and native realms are now enabled unless explicitly disabled. If -explicitly disabled, the file and native realms remain disabled at all times. - -Previously, the file and native realms had the following implicit behaviors: - -* If the file and native realms were not configured, they were implicitly disabled -if any other realm was configured. - -* If no other realm was available because realms were either not configured, -not permitted by license, or explicitly disabled, the file and native realms -were enabled, even if explicitly disabled. - -*Impact* + -To explicitly disable the file or native realm, set the respective -`file..enabled` or `native..enabled` setting to `false` -under the `xpack.security.authc.realms` namespace in `elasticsearch.yml`. - -The following configuration example disables the native realm and the file realm. - -[source,yaml] ----- -xpack.security.authc.realms: - - native.realm1.enabled: false - file.realm2.enabled: false - - ... ----- -==== - -.The realm `order` setting is now required. -[%collapsible] -==== -*Details* + -The `xpack.security.authc.realms.{type}.{name}.order` setting is now required and must be -specified for each explicitly configured realm. Each value must be unique. - -*Impact* + -The cluster will fail to start if the requirements are not met. - -For example, the following configuration is invalid: -[source,yaml] --------------------------------------------------- -xpack.security.authc.realms.kerberos.kerb1: - keytab.path: es.keytab - remove_realm_name: false --------------------------------------------------- - -And must be configured as: -[source,yaml] --------------------------------------------------- -xpack.security.authc.realms.kerberos.kerb1: - order: 0 - keytab.path: es.keytab - remove_realm_name: false --------------------------------------------------- -==== - -[[audit-logs-are-rolled-over-and-archived-by-size]] -.Audit logs are rolled-over and archived by size. -[%collapsible] -==== -*Details* + -In addition to the existing daily rollover, the security audit logs are -now rolled-over by disk size limit as well. Moreover, the rolled-over logs -are also gzip compressed. - -*Impact* + -The names of rolled over audit log files (but not the name of the current log) -have changed. -If you've set up automated tools to consume these files, you must configure them -to use the new names and to possibly account for `gzip` archives instead of -plain text. The Docker build of {es} is not affected because it logs on `stdout`, -where rollover is not performed. -==== - -[[accept-default-password-removed]] -.The `accept_default_password` setting has been removed. -[%collapsible] -==== -*Details* + -The `xpack.security.authc.accept_default_password` setting has not had any affect -since the 6.0 release of {es}. It has been removed and cannot be used. - -*Impact* + -Discontinue use of the `xpack.security.authc.accept_default_password` setting. -Specifying this setting in `elasticsearch.yml` will result in an error on -startup. -==== - -[[roles-index-cache-removed]] -.The `roles.index.cache.*` settings have been removed. -[%collapsible] -==== -*Details* + -The `xpack.security.authz.store.roles.index.cache.max_size` and -`xpack.security.authz.store.roles.index.cache.ttl` settings have -been removed. These settings have been redundant and deprecated -since the 5.2 release of {es}. - -*Impact* + -Discontinue use of the `xpack.security.authz.store.roles.index.cache.max_size` -and `xpack.security.authz.store.roles.index.cache.ttl` settings. Specifying -these settings in `elasticsearch.yml` will result in an error on startup. -==== - -[[migrate-tool-removed]] -.The `elasticsearch-migrate` tool has been removed. -[%collapsible] -==== -*Details* + -The `elasticsearch-migrate` tool provided a way to convert file -realm users and roles into the native realm. It has been deprecated -since {es} 7.2.0. Users and roles should now be created in the native -realm directly. - -*Impact* + -Discontinue use of the `elasticsearch-migrate` tool. Attempts to use the -`elasticsearch-migrate` tool will result in an error. -==== - -[[separating-node-and-client-traffic]] -.The `transport.profiles.*.xpack.security.type` setting has been removed. -[%collapsible] -==== -*Details* + -The `transport.profiles.*.xpack.security.type` setting has been removed since -the Transport Client has been removed and therefore all client traffic now uses -the HTTP transport. Transport profiles using this setting should be removed. - -*Impact* + -Discontinue use of the `transport.profiles.*.xpack.security.type` setting. -Specifying this setting in a transport profile in `elasticsearch.yml` will -result in an error on startup. -==== - -[discrete] -[[saml-realm-nameid-changes]] -.The `nameid_format` SAML realm setting no longer has a default value. -[%collapsible] -==== -*Details* + -In SAML, Identity Providers (IdPs) can either be explicitly configured to -release a `NameID` with a specific format, or configured to attempt to conform -with the requirements of a Service Provider (SP). The SP declares its -requirements in the `NameIDPolicy` element of a SAML Authentication Request. -In {es}, the `nameid_format` SAML realm setting controls the `NameIDPolicy` -value. - -Previously, the default value for `nameid_format` was -`urn:oasis:names:tc:SAML:2.0:nameid-format:transient`. This setting created -authentication requests that required the IdP to release `NameID` with a -`transient` format. - -The default value has been removed, which means that {es} will create SAML Authentication Requests by default that don't put this requirement on the -IdP. If you want to retain the previous behavior, set `nameid_format` to -`urn:oasis:names:tc:SAML:2.0:nameid-format:transient`. - -*Impact* + -If you currently don't configure `nameid_format` explicitly, it's possible -that your IdP will reject authentication requests from {es} because the requests -do not specify a `NameID` format (and your IdP is configured to expect one). -This mismatch can result in a broken SAML configuration. If you're unsure whether -your IdP is explicitly configured to use a certain `NameID` format and you want to retain current behavior -, try setting `nameid_format` to `urn:oasis:names:tc:SAML:2.0:nameid-format:transient` explicitly. -==== - -[discrete] -[[ssl-validation-changes]] -===== SSL/TLS configuration validation - -.The `xpack.security.transport.ssl.enabled` setting is now required to configure `xpack.security.transport.ssl` settings. -[%collapsible] -==== -*Details* + -It is now an error to configure any SSL settings for -`xpack.security.transport.ssl` without also configuring -`xpack.security.transport.ssl.enabled`. - -*Impact* + -If using other `xpack.security.transport.ssl` settings, you must explicitly -specify the `xpack.security.transport.ssl.enabled` setting. - -If you do not want to enable SSL and are currently using other -`xpack.security.transport.ssl` settings, do one of the following: - -* Explicitly specify `xpack.security.transport.ssl.enabled` as `false` -* Discontinue use of other `xpack.security.transport.ssl` settings - -If you want to enable SSL, follow the instructions in -{ref}/configuring-tls.html#tls-transport[Encrypting communications between nodes -in a cluster]. As part of this configuration, explicitly specify -`xpack.security.transport.ssl.enabled` as `true`. - -For example, the following configuration is invalid: -[source,yaml] --------------------------------------------------- -xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 -xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 --------------------------------------------------- - -And must be configured as: -[source,yaml] --------------------------------------------------- -xpack.security.transport.ssl.enabled: true <1> -xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 -xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 --------------------------------------------------- -<1> or `false`. -==== - -.The `xpack.security.http.ssl.enabled` setting is now required to configure `xpack.security.http.ssl` settings. -[%collapsible] -==== -*Details* + -It is now an error to configure any SSL settings for -`xpack.security.http.ssl` without also configuring -`xpack.security.http.ssl.enabled`. - -*Impact* + -If using other `xpack.security.http.ssl` settings, you must explicitly -specify the `xpack.security.http.ssl.enabled` setting. - -If you do not want to enable SSL and are currently using other -`xpack.security.http.ssl` settings, do one of the following: - -* Explicitly specify `xpack.security.http.ssl.enabled` as `false` -* Discontinue use of other `xpack.security.http.ssl` settings - -If you want to enable SSL, follow the instructions in -{ref}/configuring-tls.html#tls-http[Encrypting HTTP client communications]. As part -of this configuration, explicitly specify `xpack.security.http.ssl.enabled` -as `true`. - -For example, the following configuration is invalid: -[source,yaml] --------------------------------------------------- -xpack.security.http.ssl.certificate: elasticsearch.crt -xpack.security.http.ssl.key: elasticsearch.key -xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ] --------------------------------------------------- - -And must be configured as either: -[source,yaml] --------------------------------------------------- -xpack.security.http.ssl.enabled: true <1> -xpack.security.http.ssl.certificate: elasticsearch.crt -xpack.security.http.ssl.key: elasticsearch.key -xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ] --------------------------------------------------- -<1> or `false`. -==== - -.A `xpack.security.transport.ssl` certificate and key are now required to enable SSL for the transport interface. -[%collapsible] -==== -*Details* + -It is now an error to enable SSL for the transport interface without also configuring -a certificate and key through use of the `xpack.security.transport.ssl.keystore.path` -setting or the `xpack.security.transport.ssl.certificate` and -`xpack.security.transport.ssl.key` settings. - -*Impact* + -If `xpack.security.transport.ssl.enabled` is set to `true`, provide a -certificate and key using the `xpack.security.transport.ssl.keystore.path` -setting or the `xpack.security.transport.ssl.certificate` and -`xpack.security.transport.ssl.key` settings. If a certificate and key is not -provided, {es} will return in an error on startup. -==== - -.A `xpack.security.http.ssl` certificate and key are now required to enable SSL for the HTTP server. -[%collapsible] -==== -*Details* + -It is now an error to enable SSL for the HTTP (Rest) server without also configuring -a certificate and key through use of the `xpack.security.http.ssl.keystore.path` -setting or the `xpack.security.http.ssl.certificate` and -`xpack.security.http.ssl.key` settings. - -*Impact* + -If `xpack.security.http.ssl.enabled` is set to `true`, provide a certificate and -key using the `xpack.security.http.ssl.keystore.path` setting or the -`xpack.security.http.ssl.certificate` and `xpack.security.http.ssl.key` -settings. If certificate and key is not provided, {es} will return in an error -on startup. -==== - -[discrete] -[[ssl-misc-changes]] -===== Other SSL/TLS changes - -.PKCS#11 keystores and trustores cannot be configured in `elasticsearch.yml` -[%collapsible] -==== -*Details* + -The settings `*.ssl.keystore.type` and `*.ssl.truststore.type` no longer accept "PKCS11" as a valid type. -This applies to all SSL settings in Elasticsearch, including - -- `xpack.security.http.keystore.type` -- `xpack.security.transport.keystore.type` -- `xpack.security.http.truststore.type` -- `xpack.security.transport.truststore.type` - -As well as SSL settings for security realms, watcher and monitoring. - -Use of a PKCS#11 keystore or truststore as the JRE's default store is not affected. - -*Impact* + -If you have a PKCS#11 keystore configured within your `elasticsearch.yml` file, you must remove that -configuration and switch to a supported keystore type, or configure your PKCS#11 keystore as the -JRE default store. -==== - -[discrete] -[[builtin-users-changes]] -===== Changes to built-in users - -.The `kibana` user has been replaced by `kibana_system`. -[%collapsible] -==== -*Details* + -The `kibana` user was historically used to authenticate {kib} to {es}. -The name of this user was confusing, and was often mistakenly used to login to {kib}. -This has been renamed to `kibana_system` in order to reduce confusion, and to better -align with other built-in system accounts. - -*Impact* + -Replace any use of the `kibana` user with the `kibana_system` user. Specifying -the `kibana` user in `kibana.yml` will result in an error on startup. - -If your `kibana.yml` used to contain: -[source,yaml] --------------------------------------------------- -elasticsearch.username: kibana --------------------------------------------------- - -then you should update to use the new `kibana_system` user instead: -[source,yaml] --------------------------------------------------- -elasticsearch.username: kibana_system --------------------------------------------------- - -IMPORTANT: The new `kibana_system` user does not preserve the previous `kibana` -user password. You must explicitly set a password for the `kibana_system` user. -==== - -[discrete] -[[builtin-roles-changes]] -===== Changes to built-in roles - -.The `kibana_user` role has been renamed `kibana_admin`. -[%collapsible] -==== -*Details* + -Users who were previously assigned the `kibana_user` role should instead be assigned -the `kibana_admin` role. This role grants the same set of privileges as `kibana_user`, but has been -renamed to better reflect its intended use. - -*Impact* + -Assign users with the `kibana_user` role to the `kibana_admin` role. -Discontinue use of the `kibana_user` role. -==== - -// end::notable-breaking-changes[] - -// These are non-notable changes - -[discrete] -// This change is not notable because it should not have any impact on upgrades -// However we document it here out of an abundance of caution -[[fips-default-hash-changed]] -===== Changes to FIPS 140 mode -.When FIPS mode is enabled the default password hash is now PBKDF2_STRETCH -[%collapsible] -==== -*Details* + -If `xpack.security.fips_mode.enabled` is true (see <>), -the value of `xpack.security.authc.password_hashing.algorithm` now defaults to -`pbkdf2_stretch`. - -In earlier versions this setting would always default to `bcrypt` and a runtime -check would prevent a node from starting unless the value was explicitly set to -a "pbkdf2" variant. - -There is no change for clusters that do not enable FIPS 140 mode. - -*Impact* + -This change should not have any impact on upgraded nodes. -Any node with an explicitly configured value for the password hashing algorithm -will continue to use that configured value. -Any node that did not have an explicitly configured password hashing algorithm in -{es} 6.x or {es} 7.x would have failed to start. -==== - +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/settings.asciidoc b/docs/reference/migration/migrate_8_0/settings.asciidoc index 3d8964ed80000..e3dd6275a92db 100644 --- a/docs/reference/migration/migrate_8_0/settings.asciidoc +++ b/docs/reference/migration/migrate_8_0/settings.asciidoc @@ -4,318 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -[[search-remote-settings-removed]] -.The `search.remote.*` settings have been removed. -[%collapsible] -==== -*Details* + -In 6.5 these settings were deprecated in favor of `cluster.remote`. In 7.x we -provided automatic upgrading of these settings to their `cluster.remote` -counterparts. In 8.0.0, these settings have been removed. Elasticsearch will -refuse to start if you have these settings in your configuration or cluster -state. - -*Impact* + -Use the replacement `cluster.remote` settings. Discontinue use of the -`search.remote.*` settings. Specifying these settings in `elasticsearch.yml` -will result in an error on startup. -==== - -[[remove-pidfile]] -.The `pidfile` setting has been replaced by `node.pidfile`. -[%collapsible] -==== -*Details* + -To ensure that all settings are in a proper namespace, the `pidfile` setting was -previously deprecated in version 7.4.0 of Elasticsearch, and is removed in -version 8.0.0. Instead, use `node.pidfile`. - -*Impact* + -Use the `node.pidfile` setting. Discontinue use of the `pidfile` setting. -Specifying the `pidfile` setting in `elasticsearch.yml` will result in an error -on startup. -==== - -[[remove-processors]] -.The `processors` setting has been replaced by `node.processors`. -[%collapsible] -==== -*Details* + -To ensure that all settings are in a proper namespace, the `processors` setting -was previously deprecated in version 7.4.0 of Elasticsearch, and is removed in -version 8.0.0. Instead, use `node.processors`. - -*Impact* + -Use the `node.processors` setting. Discontinue use of the `processors` setting. -Specifying the `processors` setting in `elasticsearch.yml` will result in an -error on startup. -==== - -.The `node.processors` setting can no longer exceed the available number of processors. -[%collapsible] -==== -*Details* + -Previously it was possible to set the number of processors used to set the -default sizes for the thread pools to be more than the number of available -processors. As this leads to more context switches and more threads but without -an increase in the number of physical CPUs on which to schedule these additional -threads, the `node.processors` setting is now bounded by the number of available -processors. - -*Impact* + -If specified, ensure the value of `node.processors` setting does not exceed the -number of available processors. Setting the `node.processors` value greater than -the number of available processors in `elasticsearch.yml` will result in an -error on startup. -==== - -.The `cluster.remote.connect` setting has been removed. -[%collapsible] -==== -*Details* + -In Elasticsearch 7.7.0, the setting `cluster.remote.connect` was deprecated in -favor of setting `node.remote_cluster_client`. In Elasticsearch 8.0.0, the -setting `cluster.remote.connect` is removed. - -*Impact* + -Use the `node.remote_cluster_client` setting. Discontinue use of the -`cluster.remote.connect` setting. Specifying the `cluster.remote.connect` -setting in `elasticsearch.yml` will result in an error on startup. -==== - -.The `node.local_storage` setting has been removed. -[%collapsible] -==== -*Details* + -In Elasticsearch 7.8.0, the setting `node.local_storage` was deprecated and -beginning in Elasticsearch 8.0.0 all nodes will require local storage. Therefore, -the `node.local_storage` setting has been removed. - -*Impact* + -Discontinue use of the `node.local_storage` setting. Specifying this setting in -`elasticsearch.yml` will result in an error on startup. -==== - -.The `auth.password` setting for HTTP monitoring has been removed. -[%collapsible] -==== -*Details* + -In Elasticsearch 7.7.0, the setting `xpack.monitoring.exporters..auth.password` -was deprecated in favor of setting `xpack.monitoring.exporters..auth.secure_password`. -In Elasticsearch 8.0.0, the setting `xpack.monitoring.exporters..auth.password` is -removed. - -*Impact* + -Use the `xpack.monitoring.exporters..auth.secure_password` -setting. Discontinue use of the -`xpack.monitoring.exporters..auth.password` setting. Specifying -the `xpack.monitoring.exporters..auth.password` setting in -`elasticsearch.yml` will result in an error on startup. -==== - -.Settings used to disable basic license features have been removed. -[%collapsible] -==== -*Details* + -The following settings were deprecated in {es} 7.8.0 and have been removed -in {es} 8.0.0: - -* `xpack.enrich.enabled` -* `xpack.flattened.enabled` -* `xpack.ilm.enabled` -* `xpack.monitoring.enabled` -* `xpack.rollup.enabled` -* `xpack.slm.enabled` -* `xpack.sql.enabled` -* `xpack.transform.enabled` -* `xpack.vectors.enabled` - -These basic license features are now always enabled. - -If you have disabled ILM so that you can use another tool to manage Watcher -indices, the newly introduced `xpack.watcher.use_ilm_index_management` setting -may be set to false. - -*Impact* + -Discontinue use of the removed settings. Specifying these settings in -`elasticsearch.yml` will result in an error on startup. -==== - -.Settings used to defer cluster recovery pending a certain number of master nodes have been removed. -[%collapsible] -==== -*Details* + -The following cluster settings have been removed: - -* `gateway.expected_nodes` -* `gateway.expected_master_nodes` -* `gateway.recover_after_nodes` -* `gateway.recover_after_master_nodes` - -It is safe to recover the cluster as soon as a majority of master-eligible -nodes have joined so there is no benefit in waiting for any additional -master-eligible nodes to start. - -*Impact* + -Discontinue use of the removed settings. If needed, use -`gateway.expected_data_nodes` or `gateway.recover_after_data_nodes` to defer -cluster recovery pending a certain number of data nodes. -==== - -.You can no longer set `xpack.searchable.snapshot.shared_cache.size` on non-frozen nodes. -[%collapsible] -==== -*Details* + -Setting `xpack.searchable.snapshot.shared_cache.size` to be positive on a node -that does not have the `data_frozen` role was deprecated in {es} 7.12.0 and has -been removed in {es} 8.0.0. - -*Impact* + -{es} only allocates partially mounted indices to nodes with the `data_frozen` -role. Do not set `xpack.searchable.snapshot.shared_cache.size` on nodes without -the `data_frozen` role. Removing the setting on nodes without the `data_frozen` -role will not impact functionality. -==== - -.By default, destructive index actions do not allow wildcards. -[%collapsible] -==== -*Details* + -The default value of the setting `action.destructive_requires_name` changes from `false` -to `true` in {es} 8.0.0. - -In previous versions, the default setting allowed users to use wildcard -patterns to delete, close, or change index blocks on indices. In order -to prevent the accidental deletion of indices that happen to match a -wildcard pattern, we now require, by default, that any such destructive -operation explicitly name the indices it intends to modify. - -*Impact* + -If you would like to use wildcard patterns for destructive actions, set -`action.destructive_requires_name` to `false` using the -{ref}/cluster-update-settings.html[] cluster settings API]. -==== - -.Legacy role settings have been removed. -[%collapsible] -==== -*Details* + -The legacy role settings: - -* `node.data` -* `node.ingest` -* `node.master` -* `node.ml` -* `node.remote_cluster_client` -* `node.transform` -* `node.voting_only` - -have been removed. Instead, use the `node.roles` setting. If you were previously -using the legacy role settings on a 7.13 or later cluster, you will have a -deprecation log message on each of your nodes indicating the exact replacement -value for `node.roles`. - -*Impact* + -Discontinue use of the removed settings. Specifying these settings in -`elasticsearch.yml` will result in an error on startup. -==== - -[[system-call-filter-setting]] -.The system call filter setting has been removed. -[%collapsible] -==== -*Details* + -Elasticsearch uses system call filters to remove its ability to fork another -process. This is useful to mitigate remote code exploits. These system call -filters are enabled by default, and were previously controlled via the setting -`bootstrap.system_call_filter`. Starting in Elasticsearch 8.0, system call -filters will be required. As such, the setting `bootstrap.system_call_filter` -was deprecated in Elasticsearch 7.13.0, and is removed as of Elasticsearch -8.0.0. - -*Impact* + -Discontinue use of the removed setting. Specifying this setting in Elasticsearch -configuration will result in an error on startup. -==== - -[[tier-filter-setting]] -.Tier filtering settings have been removed. -[%collapsible] -==== -*Details* + -The cluster and index level settings ending in `._tier` used for filtering the allocation of a shard -to a particular set of nodes have been removed. Instead, the -{ref}/data-tier-shard-filtering.html#tier-preference-allocation-filter[tier -preference setting], `index.routing.allocation.include._tier_preference` should -be used. The removed settings are: - -Cluster level settings: - -- `cluster.routing.allocation.include._tier` -- `cluster.routing.allocation.exclude._tier` -- `cluster.routing.allocation.require._tier` - -Index settings: - -- `index.routing.allocation.include._tier` -- `index.routing.allocation.exclude._tier` -- `index.routing.allocation.require._tier` - -*Impact* + -Discontinue use of the removed settings. Specifying any of these cluster settings in Elasticsearch -configuration will result in an error on startup. Any indices using these settings will have the -settings archived (and they will have no effect) when the index metadata is loaded. -==== - -[[shared-data-path-setting]] -.Shared data path and per index data path settings are deprecated. -[%collapsible] -==== -*Details* + -Elasticsearch uses the shared data path as the base path of per index data -paths. This feature was previously used with shared replicas. Starting in -7.13.0, these settings are deprecated. Starting in 8.0 only existing -indices created in 7.x will be capable of using the shared data path and -per index data path settings. - -*Impact* + -Discontinue use of the deprecated settings. -==== - -[[single-data-node-watermark-setting]] -.The single data node watermark setting is deprecated and now only accepts `true`. -[%collapsible] -==== -*Details* + -In 7.14, setting `cluster.routing.allocation.disk.watermark.enable_for_single_data_node` -to false was deprecated. Starting in 8.0, the only legal value will be -true. In a future release, the setting will be removed completely, with same -behavior as if the setting was `true`. - -If the old behavior is desired for a single data node cluster, disk based -allocation can be disabled by setting -`cluster.routing.allocation.disk.threshold_enabled: false` - -*Impact* + -Discontinue use of the deprecated setting. -==== - -[[auto-import-dangling-indices-removed]] -.The `gateway.auto_import_dangling_indices` setting has been removed. -[%collapsible] -==== -*Details* + -The `gateway.auto_import_dangling_indices` cluster setting has been removed. -Previously, you could use this setting to automatically import -{ref}/modules-gateway.html#dangling-indices[dangling indices]. However, -automatically importing dangling indices is unsafe. Use the -{ref}/indices.html#dangling-indices-api[dangling indices APIs] to manage and -import dangling indices instead. - -*Impact* + -Discontinue use of the removed setting. Specifying the setting in -`elasticsearch.yml` will result in an error on startup. -==== //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/snapshots.asciidoc b/docs/reference/migration/migrate_8_0/snapshots.asciidoc index 3716a7d9b7d4e..0c161712a7514 100644 --- a/docs/reference/migration/migrate_8_0/snapshots.asciidoc +++ b/docs/reference/migration/migrate_8_0/snapshots.asciidoc @@ -4,83 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.The `repositories.fs.compress` node-level setting has been removed. -[%collapsible] -==== -*Details* + -For shared file system repositories (`"type": "fs"`), the node level setting `repositories.fs.compress` could -previously be used to enable compression for all shared file system repositories where `compress` was not specified. -The `repositories.fs.compress` setting has been removed. - -*Impact* + -Use the repository specific `compress` setting to enable compression. See -{ref}/snapshots-register-repository.html[Register a snapshot repository] for -information on the `compress` setting. - -Discontinue use of the `repositories.fs.compress` node-level setting. -==== - -.Metadata files are now compressed by default. -[%collapsible] -==== -*Details* + -Previously, the default value for `compress` was `false`. The default has been changed to `true`. - -This change will affect both newly created repositories and existing repositories where `compress=false` has not been -explicitly specified. - -For more information on the compress option, see -{ref}/snapshots-register-repository.html[Register a snapshot repository]. - -*Impact* + -Update your workflow and applications to assume a default value of `true` for -the `compress` parameter. -==== - -.The S3 repository plugin now uses a DNS-style access pattern by default. -[%collapsible] -==== -*Details* + -Starting in version 7.4 the `repository-s3` plugin does not use the -now-deprecated path-style access pattern by default. In versions 7.0, 7.1, 7.2 -and 7.3 the `repository-s3` plugin always used the path-style access pattern. -This is a breaking change for deployments that only support path-style access -but which are recognized as supporting DNS-style access by the AWS SDK. This -breaking change was made necessary by -https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/[AWS's -announcement] that the path-style access pattern is deprecated and will be -unsupported on buckets created after September 30th 2020. - -*Impact* + -If your deployment only supports path-style access and is affected by this -change then you must configure the S3 client setting `path_style_access` to -`true`. -==== - -.Restore requests no longer accept settings. -[%collapsible] -==== -*Details* + -In earlier versions, you could pass both `settings` and `index_settings` in the -body of a restore snapshot request, but the `settings` value was ignored. The -restore snapshot API now rejects requests that include a `settings` value. - -*Impact* + -Discontinue use of the `settings` parameter in restore -snapshot request. Requests that include these parameters will return an error. -==== - -.The repository stats API has been removed. -[%collapsible] -==== -*Details* + -The repository stats API has been removed. We deprecated this experimental API -in 7.10.0. - -*Impact* + -Use the {ref}/repositories-metering-apis.html[repositories metering APIs] -instead. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/threadpool.asciidoc b/docs/reference/migration/migrate_8_0/threadpool.asciidoc index b8bf4b2a51ac2..9a8d73b30679f 100644 --- a/docs/reference/migration/migrate_8_0/threadpool.asciidoc +++ b/docs/reference/migration/migrate_8_0/threadpool.asciidoc @@ -4,33 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] - -.The `listener` thread pool has been removed. -[%collapsible] -==== -*Details* + -Previously, the transport client used the thread pool to ensure listeners aren't -called back on network threads. The transport client has been removed -in 8.0, and the thread pool is no longer needed. - -*Impact* + -Remove `listener` thread pool settings from `elasticsearch.yml` for any nodes. -Specifying `listener` thread pool settings in `elasticsearch.yml` will result in -an error on startup. -==== - -.The `fixed_auto_queue_size` thread pool type has been removed. -[%collapsible] -==== -*Details* + -The `fixed_auto_queue_size` thread pool type, previously marked as an -experimental feature, was deprecated in 7.x and has been removed in 8.0. -The `search` and `search_throttled` thread pools have the `fixed` type now. - -*Impact* + -No action needed. -==== - //end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/transform.asciidoc b/docs/reference/migration/migrate_8_0/transform.asciidoc index 690b175fabf10..839b5dda73047 100644 --- a/docs/reference/migration/migrate_8_0/transform.asciidoc +++ b/docs/reference/migration/migrate_8_0/transform.asciidoc @@ -6,7 +6,7 @@ //Installation and Upgrade Guide //tag::notable-breaking-changes[] -.{transforms-cap} created in 7.4 or earlier versions must be upgraded +.{transforms-cap} created in 7.4 or earlier versions must be upgraded. [%collapsible] ==== *Details* + diff --git a/docs/reference/migration/migrate_8_0/transport.asciidoc b/docs/reference/migration/migrate_8_0/transport.asciidoc index f340212b2d8ea..2128a038342a3 100644 --- a/docs/reference/migration/migrate_8_0/transport.asciidoc +++ b/docs/reference/migration/migrate_8_0/transport.asciidoc @@ -4,77 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.Several `transport` settings have been replaced. -[%collapsible] -==== -*Details* + -The following settings have been deprecated in 7.x and removed in 8.0. Each setting has a replacement -setting that was introduced in 6.7. - -- `transport.tcp.port` replaced by `transport.port` -- `transport.tcp.compress` replaced by `transport.compress` -- `transport.tcp.connect_timeout` replaced by `transport.connect_timeout` -- `transport.tcp_no_delay` replaced by `transport.tcp.no_delay` -- `transport.profiles.profile_name.tcp_no_delay` replaced by `transport.profiles.profile_name.tcp.no_delay` -- `transport.profiles.profile_name.tcp_keep_alive` replaced by `transport.profiles.profile_name.tcp.keep_alive` -- `transport.profiles.profile_name.reuse_address` replaced by `transport.profiles.profile_name.tcp.reuse_address` -- `transport.profiles.profile_name.send_buffer_size` replaced by `transport.profiles.profile_name.tcp.send_buffer_size` -- `transport.profiles.profile_name.receive_buffer_size` replaced by `transport.profiles.profile_name.tcp.receive_buffer_size` - -*Impact* + -Use the replacement settings. Discontinue use of the removed settings. -Specifying the removed settings in `elasticsearch.yml` will result in an error -on startup. -==== - -.The `es.unsafely_permit_handshake_from_incompatible_builds` system property has been removed. -[%collapsible] -==== -*Details* + -{es} has a check that verifies that communicating pairs of nodes of the same -version are running exactly the same build and therefore using the same wire -format as each other. In previous versions this check can be bypassed by -setting the system property -`es.unsafely_permit_handshake_from_incompatible_builds` to `true`. The use of -this system property is now forbidden. - -*Impact* + -Discontinue use of the `es.unsafely_permit_handshake_from_incompatible_builds` -system property, and ensure that all nodes of the same version are running -exactly the same build. Setting this system property will result in an error -on startup. -==== - -.Selective transport compression has been enabled by default. -[%collapsible] -==== -*Details* + -Prior to 8.0, transport compression was disabled by default. Starting in 8.0, -`transport.compress` defaults to `indexing_data`. This configuration means that -the propagation of raw indexing data will be compressed between nodes. - -*Impact* + -Inter-node transit will get reduced along the indexing path. In some scenarios, -CPU usage could increase. -==== - -.Transport compression defaults to lz4. -[%collapsible] -==== -*Details* + -Prior to 8.0, the `transport.compression_scheme` setting defaulted to `deflate`. Starting in -8.0, `transport.compress_scheme` defaults to `lz4`. - -Prior to 8.0, the `cluster.remote..transport.compression_scheme` -setting defaulted to `deflate` when `cluster.remote..transport.compress` -was explicitly configured. Starting in 8.0, -`cluster.remote..transport.compression_scheme` will fallback to -`transport.compression_scheme` by default. - -*Impact* + -This configuration means that transport compression will produce somewhat lower -compression ratios in exchange for lower CPU load. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_0/watcher.asciidoc b/docs/reference/migration/migrate_8_0/watcher.asciidoc index 00ad4310535bd..fa5aed7001740 100644 --- a/docs/reference/migration/migrate_8_0/watcher.asciidoc +++ b/docs/reference/migration/migrate_8_0/watcher.asciidoc @@ -4,20 +4,5 @@ //NOTE: The notable-breaking-changes tagged regions are re-used in the //Installation and Upgrade Guide - //tag::notable-breaking-changes[] -.Watcher history now writes to a hidden data stream. -[%collapsible] -==== -*Details* + -In 8.x, {es} writes Watcher history to a hidden -`.watcher-history-` data stream. Previously, {es} wrote -Watcher history to hidden -`.watcher-history--` indices. - -*Impact* + -Update your requests to target the Watcher history data stream. For example, use -the `.watcher-history-*` wildcard expression. Requests that specifically target -non-existent Watcher history indices may return an error. -==== -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] diff --git a/docs/reference/migration/migrate_8_1.asciidoc b/docs/reference/migration/migrate_8_1.asciidoc index 482a6addf1d1c..46580e8ba7b57 100644 --- a/docs/reference/migration/migrate_8_1.asciidoc +++ b/docs/reference/migration/migrate_8_1.asciidoc @@ -11,6 +11,7 @@ See also <> and <>. coming[8.1.0] +//// [discrete] [[breaking-changes-8.1]] === Breaking changes @@ -32,7 +33,8 @@ enable <>. //tag::notable-breaking-changes[] -// end::notable-breaking-changes[] +//end::notable-breaking-changes[] +//// [discrete] [[deprecated-8.1]] @@ -49,9 +51,11 @@ the old behavior is supported until the next major release. To find out if you are using any deprecated functionality, enable <>. +//tag::notable-breaking-changes[] + [discrete] -[[breaking_8.1_settings_deprecation]] -==== Settings deprecations +[[breaking_8.1_cluster_node_setting_deprecations]] +==== Cluster and node setting deprecations [[deprecate-legacy-discovery-type-setting]] .Legacy values for the `discovery.type` setting are deprecated. @@ -69,8 +73,8 @@ discovery type. ==== [discrete] -[[breaking_8.1_crud_deprecation]] -==== CRUD deprecations +[[breaking_8.1_rest_api_deprecations]] +==== REST API deprecations [[deprecate-lenient-parsing-of-bulk-actions]] .Lenient parsing of bulk actions is deprecated. @@ -86,3 +90,4 @@ actions. Ensure that bulk actions are well-formed JSON objects containing a single entry with the correct key. ==== +//end::notable-breaking-changes[]