diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index 2cc088b8a7a..14cf2c66585 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -125,7 +125,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add MQTT input. {issue}15602[15602] {pull}16204[16204] - Add ECS categorization fields to activemq module. {issue}16151[16151] {pull}16201[16201] - Add a TLS test and more debug output to httpjson input {pull}16315[16315] -- Add an SSL config example in config.yml for filebeat MISP module. {pull}16320[16320] +- Add an SSL config example in config.yml for filebeat MISP module. {pull}16320[16320] - Improve ECS categorization, container & process field mappings in auditd module. {issue}16153[16153] {pull}16280[16280] *Heartbeat* @@ -165,6 +165,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add support for Dropwizard metrics 4.1. {pull}16332[16332] - Improve the `haproxy` module to support metrics exposed via HTTPS. {issue}14579[14579] {pull}16333[16333] - Add metricsets based on Ceph Manager Daemon to the `ceph` module. {issue}7723[7723] {pull}16254[16254] +- Release `statsd` module as GA. {pull}16447[16447] {issue}14280[14280] *Packetbeat* diff --git a/metricbeat/docs/modules/statsd.asciidoc b/metricbeat/docs/modules/statsd.asciidoc index 4eae7c058d8..491d1e71877 100644 --- a/metricbeat/docs/modules/statsd.asciidoc +++ b/metricbeat/docs/modules/statsd.asciidoc @@ -6,10 +6,41 @@ This file is generated! See scripts/mage/docs_collector.go [role="xpack"] == Statsd module -beta[] +The `statsd` module is a Metricbeat module which spawns a UDP server and listens for metrics in StatsD compatible +format. -The Statsd module is a Metricbeat module which opens a UDP port and listens for statsd metrics. +[float] +=== Metric types + +The module supports the following types of metrics: + +*Counter (c)*:: Measurement which accumulates over period of time until flushed (value set to 0). + +*Gauge (g)*:: Measurement which can increase, decrease or be set to a value. + +*Timer (ms)*:: Time measurement (in milliseconds) of an event. + +*Histogram (h)*:: Time measurement, alias for timer. + +*Set (s)*:: Measurement which counts unique occurrences until flushed (value set to 0). + +[float] +=== Module-specific configuration notes + +The `statsd` module has these additional config options: + +*`ttl`*:: It defines how long a metric will be reported after it was last recorded. +Irrespective of the given ttl, metrics will be reported at least once. +A ttl of zero means metrics will never expire. + +[float] +=== Metricsets +Currently, there is only `server` metricset in `statsd` module. + +[float] +==== `server` +The metricset collects metric data sent using UDP and publishes them under the `statsd` prefix. [float] diff --git a/metricbeat/docs/modules/statsd/server.asciidoc b/metricbeat/docs/modules/statsd/server.asciidoc index a44d850cdff..b3e983bbbd6 100644 --- a/metricbeat/docs/modules/statsd/server.asciidoc +++ b/metricbeat/docs/modules/statsd/server.asciidoc @@ -5,8 +5,6 @@ This file is generated! See scripts/mage/docs_collector.go [[metricbeat-metricset-statsd-server]] === Statsd server metricset -beta[] - include::../../../../x-pack/metricbeat/module/statsd/server/_meta/docs.asciidoc[] This is a default metricset. If the host module is unconfigured, this metricset is enabled by default. diff --git a/metricbeat/docs/modules_list.asciidoc b/metricbeat/docs/modules_list.asciidoc index bf18f416ef0..cbb56506e60 100644 --- a/metricbeat/docs/modules_list.asciidoc +++ b/metricbeat/docs/modules_list.asciidoc @@ -205,8 +205,8 @@ This file is generated! See scripts/mage/docs_collector.go .3+| .3+| |<> beta[] |<> beta[] |<> beta[] -|<> beta[] |image:./images/icon-no.png[No prebuilt dashboards] | -.1+| .1+| |<> beta[] +|<> |image:./images/icon-no.png[No prebuilt dashboards] | +.1+| .1+| |<> |<> |image:./images/icon-yes.png[Prebuilt dashboards are available] | .17+| .17+| |<> |<> diff --git a/x-pack/metricbeat/module/statsd/README.md b/x-pack/metricbeat/module/statsd/README.md index f6ea717bd97..3fe15443b02 100644 --- a/x-pack/metricbeat/module/statsd/README.md +++ b/x-pack/metricbeat/module/statsd/README.md @@ -1,37 +1,32 @@ -For manual testing and development of this module, start metricbeat with the standard configuration for the module: +### Development -``` ------------------------------------------------------------------------------- +Run metricbeat locally with configured `statsd` module: + +```yaml - module: statsd metricsets: ["server"] host: "localhost" port: "8125" enabled: true #ttl: "30s" ------------------------------------------------------------------------------- -``` - -Look for a log line to this effect: -``` -Started listening for UDP on: 127.0.0.1:8125 ``` + +Use favorite statsd client to emit metrics, e.g.: -then use a statsd client to test the features. In an empty directory do the following: - -``` +```bash $ npm install statsd-client $ node -> var SDC = require('statsd-client'), - sdc = new SDC({host: 'localhost', port: 8125}); +``` -> sdc.increment('systemname.subsystem.value'); // Increment by one -> sdc.gauge('what.you.gauge', 100); -> sdc.gaugeDelta('what.you.gauge', -70); // Will now count 50 -> sdc.gauge('gauge.with.tags', 100, {foo: 'bar'}); -> sdc.set('set.with.tags', 100, {foo: 'bar'}); -> sdc.set('set.with.tags', 200, {foo: 'bar'}); -> sdc.set('set.with.tags', 100, {foo: 'baz'}); -.... +Emit some metrics: - +```javascript +let SDC = require('statsd-client'), sdc = new SDC({host: 'localhost', port: 8125}); +sdc.increment('systemname.subsystem.value'); +sdc.gauge('what.you.gauge', 100); +sdc.gaugeDelta('what.you.gauge', -70); +sdc.gauge('gauge.with.tags', 100, {foo: 'bar'}); +sdc.set('set.with.tags', 100, {foo: 'bar'}); +sdc.set('set.with.tags', 200, {foo: 'bar'}); +sdc.set('set.with.tags', 100, {foo: 'baz'}); ``` diff --git a/x-pack/metricbeat/module/statsd/_meta/docs.asciidoc b/x-pack/metricbeat/module/statsd/_meta/docs.asciidoc index 41786036d17..22a85f0d0fd 100644 --- a/x-pack/metricbeat/module/statsd/_meta/docs.asciidoc +++ b/x-pack/metricbeat/module/statsd/_meta/docs.asciidoc @@ -1,2 +1,35 @@ -The Statsd module is a Metricbeat module which opens a UDP port and listens for statsd metrics. +The `statsd` module is a Metricbeat module which spawns a UDP server and listens for metrics in StatsD compatible +format. +[float] +=== Metric types + +The module supports the following types of metrics: + +*Counter (c)*:: Measurement which accumulates over period of time until flushed (value set to 0). + +*Gauge (g)*:: Measurement which can increase, decrease or be set to a value. + +*Timer (ms)*:: Time measurement (in milliseconds) of an event. + +*Histogram (h)*:: Time measurement, alias for timer. + +*Set (s)*:: Measurement which counts unique occurrences until flushed (value set to 0). + +[float] +=== Module-specific configuration notes + +The `statsd` module has these additional config options: + +*`ttl`*:: It defines how long a metric will be reported after it was last recorded. +Irrespective of the given ttl, metrics will be reported at least once. +A ttl of zero means metrics will never expire. + +[float] +=== Metricsets + +Currently, there is only `server` metricset in `statsd` module. + +[float] +==== `server` +The metricset collects metric data sent using UDP and publishes them under the `statsd` prefix. diff --git a/x-pack/metricbeat/module/statsd/_meta/fields.yml b/x-pack/metricbeat/module/statsd/_meta/fields.yml index a9fe3e8e931..dc97df4b5ac 100644 --- a/x-pack/metricbeat/module/statsd/_meta/fields.yml +++ b/x-pack/metricbeat/module/statsd/_meta/fields.yml @@ -2,7 +2,7 @@ title: "Statsd" description: > Statsd module - release: beta + release: ga fields: - name: statsd type: group diff --git a/x-pack/metricbeat/module/statsd/fields.go b/x-pack/metricbeat/module/statsd/fields.go index 4cc1e0c02a8..af6fe1f3a0d 100644 --- a/x-pack/metricbeat/module/statsd/fields.go +++ b/x-pack/metricbeat/module/statsd/fields.go @@ -19,5 +19,5 @@ func init() { // AssetStatsd returns asset data. // This is the base64 encoded gzipped contents of module/statsd. func AssetStatsd() string { - return "eJyUkTtuxiAQhHtOMaL5JUvOAShyiRzAwrC2SHgJ1oVvH/kVkcgpvOV8s+Jb0eOLVoXKmqsVADv2pCA/9kAKwFI1xWV2KSq8CwA4IEKyiycBFPKkKymMxFoAkyNvq9q7PaIO1LywDa+ZFOaSlnwm7Uq79ureTFoiv37ItZ3GTzLcxEcwHNSnON+zIeicXZzPotyasqne3HvNefcuRKXeyXZPRSef9D/wr2n3TDMQF2day9//9B0AAP//sz6NDg==" + return "eJyUkTtqxTAQRXut4qLGYHAWoCKbyAKMIo2NEv2QxoV3H/wL4uFXeMp7z4gzaMAvrQqVNVcrAHbsSUF+7YEUgKVqisvsUlT4FABwlAjJLp4EUMiTrqQwawFMjrytaicHRB2oeX8bXvPGlrTkM2lX2rWu/zBpidz9N9d2+v4hw018BOPR+hTn+24MOmcX5xOUGykb9Obaa86rdyEq9U62fyo6+aTflK+m/TPNQFycaS3bX/oLAAD//wYwi2Y=" } diff --git a/x-pack/metricbeat/module/statsd/server/_meta/docs.asciidoc b/x-pack/metricbeat/module/statsd/server/_meta/docs.asciidoc index 7b4e9a42db6..679bd6ea2bc 100644 --- a/x-pack/metricbeat/module/statsd/server/_meta/docs.asciidoc +++ b/x-pack/metricbeat/module/statsd/server/_meta/docs.asciidoc @@ -1,17 +1,3 @@ -This is the server metricset of the statsd module. - -Events sent to the stats endpoint will be put by default under the `statsd` prefix. - -_ttl_ defines how long a metric will be reported after it was last recorded. -Irrespective of the given ttl, metrics will be reported at least once. -A ttl of zero means metrics will never expire. - -["source","yaml",subs="attributes"] ------------------------------------------------------------------------------- -- module: statsd - metricsets: ["server"] - host: "localhost" - port: "8125" - #ttl: "30s" ------------------------------------------------------------------------------- +[role="xpack"] +This is the `server` metricset of the `statsd` module. diff --git a/x-pack/metricbeat/module/statsd/server/_meta/fields.yml b/x-pack/metricbeat/module/statsd/server/_meta/fields.yml index 05c31ead2f9..a927f3fc9f8 100644 --- a/x-pack/metricbeat/module/statsd/server/_meta/fields.yml +++ b/x-pack/metricbeat/module/statsd/server/_meta/fields.yml @@ -1 +1 @@ -- release: beta \ No newline at end of file +- release: ga diff --git a/x-pack/metricbeat/module/statsd/test_statsd.py b/x-pack/metricbeat/module/statsd/test_statsd.py new file mode 100644 index 00000000000..15d4b0513b7 --- /dev/null +++ b/x-pack/metricbeat/module/statsd/test_statsd.py @@ -0,0 +1,46 @@ +import os +import socket +import sys + +sys.path.append(os.path.join(os.path.dirname(__file__), '../../tests/system')) +from xpack_metricbeat import XPackTest, metricbeat + +STATSD_HOST = '127.0.0.1' +STATSD_PORT = 8125 + +METRIC_MESSAGE = bytes('metric1:777.0|g|#k1:v1,k2:v2', 'utf-8') + + +class Test(XPackTest): + + def test_server(self): + """ + statsd server metricset test + """ + + # Start the application + self.render_config_template(modules=[{ + "name": "statsd", + "metricsets": ["server"], + "period": "5s", + "host": STATSD_HOST, + "port": STATSD_PORT, + }]) + proc = self.start_beat() + self.wait_until(lambda: self.log_contains("Started listening for UDP")) + + # Send UDP packet with metric + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + sock.sendto(METRIC_MESSAGE, (STATSD_HOST, STATSD_PORT)) + sock.close() + + self.wait_until(lambda: self.output_lines() > 0) + proc.check_kill_and_wait() + self.assert_no_logged_warnings(replace='use of closed network connection') + + # Verify output + output = self.read_output_json() + self.assertGreater(len(output), 0) + evt = output[0] + assert evt["statsd"]["metric1"]["value"] == 777 + self.assert_fields_are_documented(evt)