Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Socket summary module #6782

Merged
merged 6 commits into from
Aug 5, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ https://github.com/elastic/beats/compare/v6.4.0...master[Check the HEAD diff]
- Move common kafka fields (broker, topic and partition.id) to the module level to facilitate events correlation {pull}7767[7767]
- Add `metircs` metricset to MongoDB module. {pull}7611[7611]
- Add fields for mermory fragmentation, memory allocator stats, copy on write, master-slave status, and active defragmentation to `info` metricset of Redis module. {pull}7695[7695]
- Add experimental socket summary metricset to system module {pull}6782[6782]

*Packetbeat*

Expand Down
92 changes: 92 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19476,6 +19476,98 @@ type: keyword
Name of the user running the process.


--

[float]
== socket.summary fields

Summary metrics of open sockets in the host system



[float]
== all fields

All connections



*`system.socket.summary.all.count`*::
+
--
type: integer

All open connections


--

*`system.socket.summary.all.listening`*::
+
--
type: integer

All listening ports


--

[float]
== tcp fields

All TCP connections



[float]
== all fields

All TCP connections



*`system.socket.summary.tcp.all.count`*::
+
--
type: integer

All open TCP connections


--

*`system.socket.summary.tcp.all.listening`*::
+
--
type: integer

All TCP listening ports


--

[float]
== udp fields

All UDP connections



[float]
== all fields

All UDP connections



*`system.socket.summary.udp.all.count`*::
+
--
type: integer

All open TCP connections


--

[float]
Expand Down
4 changes: 4 additions & 0 deletions metricbeat/docs/modules/system.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@ The following metricsets are available:

* <<metricbeat-metricset-system-socket,socket>>

* <<metricbeat-metricset-system-socket_summary,socket_summary>>

* <<metricbeat-metricset-system-uptime,uptime>>

include::system/core.asciidoc[]
Expand All @@ -157,5 +159,7 @@ include::system/raid.asciidoc[]

include::system/socket.asciidoc[]

include::system/socket_summary.asciidoc[]

include::system/uptime.asciidoc[]

17 changes: 17 additions & 0 deletions metricbeat/docs/modules/system/socket_summary.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
////
This file is generated! See scripts/docs_collector.py
////

[[metricbeat-metricset-system-socket_summary]]
=== System socket_summary metricset

experimental[]

include::../../../module/system/socket_summary/_meta/docs.asciidoc[]


==== Fields

For a description of each field in the metricset, see the
<<exported-fields-system,exported fields>> section.

3 changes: 2 additions & 1 deletion metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ This file is generated! See scripts/docs_collector.py
.2+| .2+| |<<metricbeat-metricset-redis-info,info>>
|<<metricbeat-metricset-redis-keyspace,keyspace>>
|<<metricbeat-module-system,System>> |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.13+| .13+| |<<metricbeat-metricset-system-core,core>>
.14+| .14+| |<<metricbeat-metricset-system-core,core>>
|<<metricbeat-metricset-system-cpu,cpu>>
|<<metricbeat-metricset-system-diskio,diskio>>
|<<metricbeat-metricset-system-filesystem,filesystem>>
Expand All @@ -131,6 +131,7 @@ This file is generated! See scripts/docs_collector.py
|<<metricbeat-metricset-system-process_summary,process_summary>>
|<<metricbeat-metricset-system-raid,raid>>
|<<metricbeat-metricset-system-socket,socket>>
|<<metricbeat-metricset-system-socket_summary,socket_summary>> experimental[]
|<<metricbeat-metricset-system-uptime,uptime>>
|<<metricbeat-module-traefik,traefik>> beta[] |image:./images/icon-no.png[No prebuilt dashboards] |
.1+| .1+| |<<metricbeat-metricset-traefik-health,health>> experimental[]
Expand Down
1 change: 1 addition & 0 deletions metricbeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion metricbeat/module/system/fields.go

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions metricbeat/module/system/socket_summary/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
The System `socket_summary` metricset provides the summary of open network sockets in the host system.

The following metrics are exported:
* TCP
* all: All connections
* listening: All listening connections
* UDP:
* all: All connections

This metricset is available on:

- FreeBSD
- Linux
- macOS
- Windows

[float]
=== Configuration

There are no configuration options for this metricset.
53 changes: 53 additions & 0 deletions metricbeat/module/system/socket_summary/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
- name: socket.summary
title: Socket summary
type: group
description: >
Summary metrics of open sockets in the host system
fields:
- name: all
type: group
description: >
All connections
fields:
- name: count
type: integer
description: >
All open connections
- name: listening
type: integer
description: >
All listening ports
- name: tcp
type: group
description: >
All TCP connections
fields:
- name: all
type: group
description: >
All TCP connections
fields:
- name: count
type: integer
description: >
All open TCP connections
- name: listening
type: integer
description: >
All TCP listening ports
- name: udp
type: group
description: >
All UDP connections
fields:
- name: all
type: group
description: >
All UDP connections
fields:
- name: count
type: integer
description: >
All open TCP connections


119 changes: 119 additions & 0 deletions metricbeat/module/system/socket_summary/socket_summary.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package socket_summary

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use an underscore in package name


import (
"syscall"

"github.com/elastic/beats/libbeat/common"

"github.com/elastic/beats/libbeat/common/cfgwarn"
"github.com/elastic/beats/metricbeat/mb"

"github.com/shirou/gopsutil/net"
)

// init registers the MetricSet with the central registry as soon as the program
// starts. The New function will be called later to instantiate an instance of
// the MetricSet for each host defined in the module's configuration. After the
// MetricSet has been created then Fetch will begin to be called periodically.
func init() {
mb.Registry.MustAddMetricSet("system", "socket_summary", New,
mb.WithNamespace("system.socket.summary"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ruflin can this cause some kind of collision with the socket metricset or with its fields definition?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case we would also use summary in the socket metricset it would so we shouldn't. We do the same trick in some other metricsets to make it easier for the user to access the metrics.

Should we create a collision on day by accident, CI should detect multiple definitions of the same field.

)
}

// MetricSet holds any configuration or state information. It must implement
// the mb.MetricSet interface. And this is best achieved by embedding
// mb.BaseMetricSet because it implements all of the required mb.MetricSet
// interface methods except for Fetch.
type MetricSet struct {
mb.BaseMetricSet
}

// New creates a new instance of the MetricSet. New is responsible for unpacking
// any MetricSet specific configuration options if there are any.
func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
cfgwarn.Experimental("The socket_summary metricset is experimental.")

return &MetricSet{
BaseMetricSet: base,
}, nil
}

func calculateConnStats(conns []net.ConnectionStat) common.MapStr {
var (
allConns = len(conns)
allListening = 0
tcpConns = 0
tcpListening = 0
udpConns = 0
)

for _, conn := range conns {
if conn.Status == "LISTEN" {
allListening++
}
switch conn.Type {
case syscall.SOCK_STREAM:
tcpConns++

if conn.Status == "LISTEN" {
tcpListening++
}
case syscall.SOCK_DGRAM:
udpConns++
}
}

return common.MapStr{
"all": common.MapStr{
"count": allConns,
"listening": allListening,
},
"tcp": common.MapStr{
"all": common.MapStr{
"count": tcpConns,
"listening": tcpListening,
},
},
"udp": common.MapStr{
"all": common.MapStr{
"count": udpConns,
},
},
}
}

// Fetch methods implements the data gathering and data conversion to the right
// format. It publishes the event which is then forwarded to the output. In case
// of an error set the Error field of mb.Event or simply call report.Error().
func (m *MetricSet) Fetch(report mb.ReporterV2) {

// all network connections
conns, err := net.Connections("inet")

if err != nil {
report.Error(err)
return
}

report.Event(mb.Event{
MetricSetFields: calculateConnStats(conns),
})
}
Loading