Skip to content

Commit

Permalink
Autoscaling APIs not experimental
Browse files Browse the repository at this point in the history
We no longer regard the autoscaling APIs experimental though they are
only intended for use by ESS/ECE/ECK. This commit updates the docs
to reflect this and adds a minimal set of documentation for the
feature.
  • Loading branch information
henningandersen committed Jan 11, 2021
1 parent a9e8a6a commit 3fbf3b2
Show file tree
Hide file tree
Showing 12 changed files with 413 additions and 13 deletions.
4 changes: 2 additions & 2 deletions docs/reference/autoscaling/apis/autoscaling-apis.asciidoc
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[role="xpack"]
[testenv="platinum"]
[testenv="enterprise"]
[[autoscaling-apis]]
== Autoscaling APIs

experimental[]
include::../autoscaling-designed-for-note.asciidoc[]

You can use the following APIs to perform autoscaling operations.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
[role="xpack"]
[testenv="platinum"]
[testenv="enterprise"]
[[autoscaling-delete-autoscaling-policy]]
=== Delete autoscaling policy API
++++
<titleabbrev>Delete autoscaling policy</titleabbrev>
++++

experimental[]
include::../autoscaling-designed-for-note.asciidoc[]

Delete autoscaling policy.

Expand Down
187 changes: 185 additions & 2 deletions docs/reference/autoscaling/apis/get-autoscaling-capacity.asciidoc
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
[role="xpack"]
[testenv="platinum"]
[testenv="enterprise"]
[[autoscaling-get-autoscaling-capacity]]
=== Get autoscaling capacity API
++++
<titleabbrev>Get autoscaling capacity</titleabbrev>
++++

experimental[]
include::../autoscaling-designed-for-note.asciidoc[]

Get autoscaling capacity.

Expand All @@ -33,6 +33,189 @@ This API gets the current autoscaling capacity based on the configured
autoscaling policy. This API will return information to size the cluster
appropriately to the current workload.

The `required_capacity` is calculated as the max of the `required_capacity`
result of all individual deciders that are enabled for the policy.

The operator should verify that the `current_nodes` match
the operator's knowledge of the cluster to avoid making autoscaling decisions
based on stale or incomplete information.

The response contains decider specific information, helping to diagnose how/why
autoscaling determined a certain capacity was required. This information should
not be relied upon for making autoscaling decisions, it's intended use is only
for diagnosing the autoscaling decision.

[role="child_attributes"]
[[autoscaling-get-autoscaling-capacity-api-response-body]]
==== {api-response-body-title}

`<policy_name>`::
(object)
Contains capacity for the policy
+
.Properties of `<policy_name>`
[%collapsible%open]
====
`required_capacity`::
(object)
Contains the required capacity for the policy
+
.Properties of `required_capacity`
[%collapsible%open]
=====
`node`::
(object)
Contains the minimum node sizes required per node (ensuring that individual
shards or ML jobs can fit into a single node)
+
.Properties of `node`
[%collapsible%open]
======
`storage`::
(integer)
Contains the amount of storage required per node, in bytes
`memory`::
(integer)
Contains the amount of memory required per node, in bytes
======
`total`::
(object)
Contains the total size required for the policy
+
.Properties of `total`
[%collapsible%open]
======
`storage`::
(integer)
Contains the amount of storage required total for the policy, in bytes
`memory`::
(integer)
Contains the amount of memory required total for the policy, in bytes
======
=====
`current_capacity`::
(object)
Contains the current capacity for nodes governed by the policy, i.e. the nodes
that {es} bases its calculation on.
+
.Properties of `current_capacity`
[%collapsible%open]
=====
`node`::
(object)
Contains the maximum sizes of nodes governed by the policy
+
.Properties of `node`
[%collapsible%open]
======
`storage`::
(integer)
Contains the maximum current storage of a node, in bytes
`memory`::
(integer)
Contains the maximum current memory of a node, in bytes
======
`total`::
(object)
Contains the current total sizes for nodes governed by the policy
+
.Properties of `total`
[%collapsible%open]
======
`storage`::
(integer)
Contains the current total amount of storage available for the policy, in bytes
`memory`::
(integer)
Contains the current total amount of memory available for the policy, in bytes
======
=====
`current_nodes`::
(array of objects)
List of current nodes that {es} used as basis for the capacity calculation
+
.Properties of elements in `current_nodes`
[%collapsible%open]
=====
`name`::
(string)
The name of the node
=====
`deciders`::
(object)
The capacity results from individual deciders, allowing insight into how the
outer level `required_capacity` was calculated.
+
.Properties of `deciders`
[%collapsible%open]
======
`<decider_name>`::
(object)
The capacity result for a specific decider that is enabled for the policy
+
.Properties of `<decider_name>`
[%collapsible%open]
=======
`required_capacity`::
(object)
The required capacity determined by the decider
+
.Properties of `required_capacity`
[%collapsible%open]
========
`node`::
(object)
Contains the minimum node sizes required per node (ensuring that individual
shards or ML jobs can fit into a single node)
+
.Properties of `node`
[%collapsible%open]
=========
`storage`::
(integer)
Contains the amount of storage required per node, in bytes
`memory`::
(integer)
Contains the amount of memory required per node, in bytes
=========
`total`::
(object)
Contains the total size required for the policy
+
.Properties of `total`
[%collapsible%open]
=========
`storage`::
(integer)
Contains the amount of storage required total for the policy, in bytes
`memory`::
(integer)
Contains the amount of memory required total for the policy, in bytes
=========
========
`reason_summary`::
(string)
A textual description of the basis for the decider's result
`reason_details`::
(object)
A per decider structure containing details forming the basis for the deciders' result.
The contents should not be relied on for application purposes and are not subject
to normal backwards compatibility guarantees.
=======
======
====


[[autoscaling-get-autoscaling-capacity-examples]]
==== {api-examples-title}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
[role="xpack"]
[testenv="platinum"]
[testenv="enterprise"]
[[autoscaling-get-autoscaling-policy]]
=== Get autoscaling policy API
++++
<titleabbrev>Get autoscaling policy</titleabbrev>
++++

experimental[]
include::../autoscaling-designed-for-note.asciidoc[]

Get autoscaling policy.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
[role="xpack"]
[testenv="platinum"]
[testenv="enterprise"]
[[autoscaling-put-autoscaling-policy]]
=== Put autoscaling policy API
++++
<titleabbrev>Put autoscaling policy</titleabbrev>
++++

experimental[]
include::../autoscaling-designed-for-note.asciidoc[]

Put autoscaling policy.

Expand Down Expand Up @@ -47,6 +47,7 @@ DELETE /_autoscaling/policy/name
==== {api-description-title}

This API puts an autoscaling policy with the provided name.
See <<autoscaling-deciders,Autoscaling Deciders>> for available deciders.

[[autoscaling-put-autoscaling-policy-examples]]
==== {api-examples-title}
Expand Down
24 changes: 24 additions & 0 deletions docs/reference/autoscaling/autoscaling-deciders.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[role="xpack"]
[testenv="enterprise"]
[[autoscaling-deciders]]
== Autoscaling deciders

<<autoscaling-reactive-storage-decider,Reactive storage decider>>::
Estimate required storage capacity of current data set. Available for policies
governing data nodes.

<<autoscaling-proactive-storage-decider,Proactive storage decider>>::
Estimate required storage capacity based on current ingestion into hot nodes.
Available for policies governing hot data nodes.

<<autoscaling-machine-learning-decider,Machine learning decider>>::
Estimate required memory capacity based on machine learning jobs.
Available for policies governing machine learning nodes.

<<autoscaling-fixed-decider,Fixed decider>>::
A decider useful for testing, responding a fixed required capacity.

include::deciders/reactive-storage-decider.asciidoc[]
include::deciders/proactive-storage-decider.asciidoc[]
include::deciders/machine-learning-decider.asciidoc[]
include::deciders/fixed-decider.asciidoc[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[NOTE]
Autoscaling is designed for indirect use by ECE/ESS and ECK. Direct use is not supported.
62 changes: 62 additions & 0 deletions docs/reference/autoscaling/deciders/fixed-decider.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
[role="xpack"]
[testenv="enterprise"]
[[autoscaling-fixed-decider]]
=== Fixed decider

experimental[]
[NOTE]
The purpose of this decider is purely for testing, it is not intended for
production use.

The `fixed` decider responds with a fixed required capacity. It is not enabled
by default but can be enabled for any policy by explicitly configuring it.

It has three configuration options:

* storage: the node level storage necessary, default -1 (disabled).
* memory: the node level memory necessary, default -1 (disabled).
* nodes: the number of nodes to use for calculating total required capacities,
default 1.

[[autoscaling-fixed-decider-examples]]
==== {api-examples-title}

This example puts an autoscaling policy named `my_autoscaling_policy`, enabling
and configuring the fixed decider.

[source,console]
--------------------------------------------------
PUT /_autoscaling/policy/my_autoscaling_policy
{
"roles" : [ "data_hot" ],
"deciders": {
"fixed": {
"storage": "1tb",
"memory": "32gb",
"nodes": 8
}
}
}
--------------------------------------------------
// TEST

The API returns the following result:

[source,console-result]
--------------------------------------------------
{
"acknowledged": true
}
--------------------------------------------------

//////////////////////////
[source,console]
--------------------------------------------------
DELETE /_autoscaling/policy/my_autoscaling_policy
--------------------------------------------------
// TEST[continued]
//////////////////////////


Loading

0 comments on commit 3fbf3b2

Please sign in to comment.