Skip to content

Commit

Permalink
CONSOLEify percentile and percentile-ranks docs
Browse files Browse the repository at this point in the history 
Related #18160
  • Loading branch information
@polyfractal
polyfractal committed Aug 2, 2017
1 parent 268923e commit d8414ff
Show file tree
Hide file tree
Showing 3 changed files with 136 additions and 59 deletions.
34 changes: 32 additions & 2 deletions docs/build.gradle
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ buildRestTests.expectedUnconvertedCandidates = [
'reference/aggregations/bucket/significantterms-aggregation.asciidoc',
'reference/aggregations/bucket/terms-aggregation.asciidoc',
'reference/aggregations/matrix/stats-aggregation.asciidoc',
'reference/aggregations/metrics/percentile-aggregation.asciidoc',
'reference/aggregations/metrics/percentile-rank-aggregation.asciidoc',
'reference/aggregations/metrics/scripted-metric-aggregation.asciidoc',
'reference/aggregations/metrics/tophits-aggregation.asciidoc',
'reference/cluster/allocation-explain.asciidoc',
Expand Down Expand Up @@ -476,3 +474,35 @@ buildRestTests.setups['analyze_sample'] = '''
properties:
obj1.field1:
type: text'''

// Used by percentile/percentile-rank aggregations
buildRestTests.setups['latency'] = '''
- do:
indices.create:
index: latency
body:
settings:
number_of_shards: 1
number_of_replicas: 1
mappings:
data:
properties:
load_time:
type: long
- do:
bulk:
index: latency
type: data
refresh: true
body: |'''


for (int i = 0; i < 100; i++) {
def value = i
if (i % 10) {
value = i*10
}
buildRestTests.setups['latency'] += """
{"index":{}}
{"load_time": "$value"}"""
}
82 changes: 54 additions & 28 deletions docs/reference/aggregations/metrics/percentile-aggregation.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@ Let's look at a range of percentiles representing load time:

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
Expand All @@ -36,6 +38,8 @@ Let's look at a range of percentiles representing load time:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency]
<1> The field `load_time` must be a numeric field

By default, the `percentile` metric will generate a range of
Expand All @@ -49,18 +53,19 @@ percentiles: `[ 1, 5, 25, 50, 75, 95, 99 ]`. The response will look like this:
"aggregations": {
"load_time_outlier": {
"values" : {
"1.0": 15,
"5.0": 20,
"25.0": 23,
"50.0": 25,
"75.0": 29,
"95.0": 60,
"99.0": 150
"1.0": 9.9,
"5.0": 29.500000000000004,
"25.0": 167.5,
"50.0": 445.0,
"75.0": 722.5,
"95.0": 940.5,
"99.0": 980.1000000000001
}
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]

As you can see, the aggregation will return a calculated value for each percentile
in the default range. If we assume response times are in milliseconds, it is
Expand All @@ -73,7 +78,9 @@ must be a value between 0-100 inclusive):

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
Expand All @@ -84,6 +91,8 @@ must be a value between 0-100 inclusive):
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency]
<1> Use the `percents` parameter to specify particular percentiles to calculate

==== Keyed Response
Expand All @@ -92,20 +101,21 @@ By default the `keyed` flag is set to `true` which associates a unique string ke

[source,js]
--------------------------------------------------
POST bank/account/_search?size=0
GET latency/data/_search
{
"size": 0,
"aggs": {
"balance_outlier": {
"load_time_outlier": {
"percentiles": {
"field": "balance",
"field": "load_time",
"keyed": false
}
}
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:bank]
// TEST[setup:latency]

Response:

Expand All @@ -115,49 +125,42 @@ Response:
...
"aggregations": {
"balance_outlier": {
"load_time_outlier": {
"values": [
{
"key": 1.0,
"value": 1462.8400000000001
"value": 9.9
},
{
"key": 5.0,
"value": 3591.85
"value": 29.500000000000004
},
{
"key": 25.0,
"value": 13709.333333333334
"value": 167.5
},
{
"key": 50.0,
"value": 26020.11666666667
"value": 445.0
},
{
"key": 75.0,
"value": 38139.648148148146
"value": 722.5
},
{
"key": 95.0,
"value": 47551.549999999996
"value": 940.5
},
{
"key": 99.0,
"value": 49339.16
"value": 980.1000000000001
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]
// TESTRESPONSE[s/1462.8400000000001/$body.aggregations.balance_outlier.values.0.value/]
// TESTRESPONSE[s/3591.85/$body.aggregations.balance_outlier.values.1.value/]
// TESTRESPONSE[s/13709.333333333334/$body.aggregations.balance_outlier.values.2.value/]
// TESTRESPONSE[s/26020.11666666667/$body.aggregations.balance_outlier.values.3.value/]
// TESTRESPONSE[s/38139.648148148146/$body.aggregations.balance_outlier.values.4.value/]
// TESTRESPONSE[s/47551.549999999996/$body.aggregations.balance_outlier.values.5.value/]
// TESTRESPONSE[s/49339.16/$body.aggregations.balance_outlier.values.6.value/]

==== Script

Expand All @@ -167,7 +170,9 @@ a script to convert them on-the-fly:

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
Expand All @@ -183,6 +188,9 @@ a script to convert them on-the-fly:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency]

<1> The `field` parameter is replaced with a `script` parameter, which uses the
script to generate values which percentiles are calculated on
<2> Scripting supports parameterized input just like any other script
Expand All @@ -191,21 +199,25 @@ This will interpret the `script` parameter as an `inline` script with the `painl

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
"script" : {
"id": "my_script",
"params" : {
"timeUnit" : 1000
"params": {
"field": "load_time"
}
}
}
}
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency,stored_example_script]

[[search-aggregations-metrics-percentile-aggregation-approximation]]
==== Percentiles are (usually) approximate
Expand Down Expand Up @@ -252,7 +264,9 @@ This balance can be controlled using a `compression` parameter:

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
Expand All @@ -265,6 +279,9 @@ This balance can be controlled using a `compression` parameter:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency]

<1> Compression controls memory usage and approximation error

The TDigest algorithm uses a number of "nodes" to approximate percentiles -- the
Expand Down Expand Up @@ -298,7 +315,9 @@ The HDR Histogram can be used by specifying the `method` parameter in the reques

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
Expand All @@ -312,6 +331,9 @@ The HDR Histogram can be used by specifying the `method` parameter in the reques
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency]

<1> `hdr` object indicates that HDR Histogram should be used to calculate the percentiles and specific settings for this algorithm can be specified inside the object
<2> `number_of_significant_value_digits` specifies the resolution of values for the histogram in number of significant digits

Expand All @@ -326,7 +348,9 @@ had a value.

[source,js]
--------------------------------------------------
GET latency/data/_search
{
"size": 0,
"aggs" : {
"grade_percentiles" : {
"percentiles" : {
Expand All @@ -337,5 +361,7 @@ had a value.
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:latency]

<1> Documents without a value in the `grade` field will fall into the same bucket as documents that have the value `10`.
Loading

0 comments on commit d8414ff

Please sign in to comment.