[DOCS] Add throttling based on configuration parameter #56653
Conversation
lcawl
added
>docs
|
Pinging @elastic/es-docs (>docs) |
|
Pinging @elastic/ml-core (:ml/Transform) |
| data it processes. Also consider whether the cluster has sufficient resources in | ||
| place to support both the composite aggregation search and the indexing of its | ||
| results. You can also throttle the rate at which {transforms} perform search and | ||
| index requests by setting `docs_per_second` when you <<put-transform,create>> or |
There was a problem hiding this comment.
and index requests
Is this correct that indexing is also affected by this setting?
There was a problem hiding this comment.
yes, by slowing down the searches, you slow down indexing, too.
Maybe add a sentence that the transform will take longer if throttling is used, although this seems obvious?
and maybe something like:
To know the current docs_per_second transform is operating you can take a look into stats (-> link to stats) and calculate the rate with:
docs_per_second = documents_processed / search_time_in_ms * 1000
(^ I plan to add this to the stats endpoint, however not for 7.8)
There was a problem hiding this comment.
Thanks, I've added that information in a final paragraph.
| data it processes. Also consider whether the cluster has sufficient resources in | ||
| place to support both the composite aggregation search and the indexing of its | ||
| results. You can also throttle the rate at which {transforms} perform search and | ||
| index requests by setting `docs_per_second` when you <<put-transform,create>> or |
There was a problem hiding this comment.
yes, by slowing down the searches, you slow down indexing, too.
Maybe add a sentence that the transform will take longer if throttling is used, although this seems obvious?
and maybe something like:
To know the current docs_per_second transform is operating you can take a look into stats (-> link to stats) and calculate the rate with:
docs_per_second = documents_processed / search_time_in_ms * 1000
(^ I plan to add this to the stats endpoint, however not for 7.8)
| @@ -128,6 +128,23 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by] | |||
| ==== | |||
| //End pivot | |||
|
|
|||
| //Begin settings | |||
There was a problem hiding this comment.
👍 sorry for the bad placement! I see, it should be ordered, will do better next time!
|
Still LGTM! |
Related to #54862
This PR adds throttling to the transform performance considerations in the transform overview (https://www.elastic.co/guide/en/elasticsearch/reference/master/transform-overview.html#transform-performance). Since there were similar performance tips in "How transforms work" (https://www.elastic.co/guide/en/elasticsearch/reference/master/transform-checkpoints.html), I've merged the two and linked them.
This PR also edits the API parameter description for docs_per_second, which was added in #56178. It also sorts the order of the parameters in the create and update transform APIs.
Preview: