Skip to content

Commit

Permalink
doc: Initialize documentation
Browse files Browse the repository at this point in the history
Both structure and huge parts are copied from Icinga DB and were altered
afterwards. The already existing Channel Plugin documentation was
extended.
  • Loading branch information
oxzi committed Jun 25, 2024
1 parent bc3910f commit d0aaedc
Show file tree
Hide file tree
Showing 8 changed files with 368 additions and 2 deletions.
35 changes: 35 additions & 0 deletions doc/01-About.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Icinga Notifications

Icinga Notifications is a set of components that processes received events from various sources, manages incidents and
forwards notifications to predefined contacts, consisting of:

* The Icinga Notifications daemon (this project), which receives events and sends notifications.
* An [Icinga Web module](https://github.com/Icinga/icinga-notifications-web) that provides graphical configuration and further processing of the data collected by the daemon.
* And Icinga 2 and other custom sources that propagate state updates and acknowledgement events to the daemon.

## Big Picture

Because Icinga Notifications consists of several components,
this section tries to help understand how these components relate.

First, the Icinga Notifications configuration resides in a SQL database.
It can be conveniently tweaked via Icinga Notifications Web directly from a web browser.
The Icinga Notifications daemon uses this database to read the current configuration.

As in any Icinga setup, all host and service checks are defined in Icinga 2.
By querying the Icinga 2 API, the Icinga Notifications daemon retrieves state changes, acknowledgements, and other events.
These events are stored in the database and are available for further inspection in the Icinga Notifications Web.

Depending on its configuration, the daemon will take further action on these events.
This optionally includes escalations that are sent through a channel plugin.
Each such channel plugin implements a domain-specific transport, e.g., the `email` channel sends emails via SMTP.
When configured, Icinga Notifications will use channel plugins to notify end users or talk to other APIs.

## Installation

To install Icinga Notifications see [Installation](02-Installation.md).

## License

Icinga Notifications and the Icinga Notifications documentation are licensed under the terms of the
GNU General Public License Version 2.
121 changes: 121 additions & 0 deletions doc/02-Installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
<!-- {% if index %} -->
# Installing Icinga Notifications

The recommended way to install Icinga Notifications is to use prebuilt packages
for all supported platforms from our official release repository.
Please follow the steps listed for your target operating system,
which guide you through setting up the repository and installing Icinga Notifications.

To upgrade an existing Icinga Notifications installation to a newer version,
see the [Upgrading](04-Upgrading.md) documentation for the necessary steps.

<!-- {% else %} -->

## Setting up the Database

A MySQL (≥5.5), MariaDB (≥10.1), or PostgreSQL (≥9.6) database is required to run Icinga Notifications.
Please follow the steps listed for your target database,
which guide you through setting up the database and user and importing the schema.

### Setting up a MySQL or MariaDB Database

If you use a version of MySQL < 5.7 or MariaDB < 10.2, the following server options must be set:

```
innodb_file_format=barracuda
innodb_file_per_table=1
innodb_large_prefix=1
```

Set up a MySQL database for Icinga Notifications:

```
# mysql -u root -p
CREATE DATABASE notifications;
CREATE USER 'notifications'@'localhost' IDENTIFIED BY 'CHANGEME';
GRANT ALL ON notifications.* TO 'notifications'@'localhost';
```

After creating the database, import the Icinga Notifications schema using the following command:

```
mysql -u root -p notifications < /usr/share/icinga-notifications/schema/mysql/schema.sql
```

### Setting up a PostgreSQL Database

Set up a PostgreSQL database for Icinga Notifications:

```
# su -l postgres
createuser -P notifications
createdb -E UTF8 --locale en_US.UTF-8 -T template0 -O notifications notifications
echo 'CREATE EXTENSION IF NOT EXISTS citext;' | psql notifications
```

The `CREATE EXTENSION` command requires the `postgresql-contrib` package.

Edit `pg_hba.conf`, insert the following before everything else:

```
local all notifications md5
host all notifications 0.0.0.0/0 md5
host all notifications ::/0 md5
```

To apply these changes, run `systemctl reload postgresql`.

After creating the database, import the Icinga Notifications schema using the following command:

```
psql -U icinga-notifications notifications < /usr/share/icinga-notifications/schema/pgsql/schema.sql
```

## Configuring Icinga Notifications

Icinga Notifications installs its configuration file to `/etc/icinga-notifications/config.yml`,
pre-populating most of the settings for a local setup. Before running Icinga Notifications,
adjust the database credentials and the Icinga Web 2 URL.
The configuration file explains general settings.
All available settings can be found under [Configuration](03-Configuration.md).

## Configuring Icinga 2

Icinga Notifications utilizes the [Icinga 2 API](https://icinga.com/docs/icinga-2/latest/doc/12-icinga2-api/)
to fetch events from an Icinga 2 instance.
Thus, if the Icinga 2 API is not already configured,
please follow the [API setup guide](https://icinga.com/docs/icinga-2/latest/doc/12-icinga2-api/#setting-up-the-api) now.
Furthermore, an [`ApiUser`](https://icinga.com/docs/icinga-2/latest/doc/09-object-types/#objecttype-apiuser)
with [permissions](https://icinga.com/docs/icinga-2/latest/doc/12-icinga2-api/#icinga2-api-permissions)
as listed in the example below is required.

```
object ApiUser "notifications" {
password = "insecure"
permissions = [ "events/*", "objects/query/*" ]
}
```

After completing setting up Icinga Notifications Web, the Icinga 2 API can be configured with this `ApiUser` as a source.

## Running Icinga Notifications

The `icinga-notifications` package automatically installs the necessary systemd unit files to run Icinga Notifications.
Please run the following command to enable and start its service:

```
systemctl enable --now icinga-notifications
```

## Installing Icinga Notifications Web

With Icinga 2, Icinga Notifications and the database fully set up, it is now time to install Icinga Notifications Web,
which connects to the database and allows configuring Icinga Notifications.

The Icinga Notifications Web package is also included in the Icinga repository, and since it is already set up,
you have completed the instructions here and can proceed to
[install Icinga Notifications Web](https://icinga.com/docs/icinga-notifications-web/latest/doc/02-Installation/).

<!-- {% endif %} --><!-- {# end else if index #} -->
22 changes: 22 additions & 0 deletions doc/02-Installation.md.d/From-Source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Installing Icinga Notifications from Source

To install Icinga Notifications and get started, you first need to clone the git repository.
```bash
git clone https://github.com/Icinga/icinga-notifications.git
cd icinga-notifications
```

Now you can build the daemon and all its channels with the `Makefile` and you will find all build artifacts in the `build` directory.
```bash
make
```

Finally, the `Makefile` also allows you to perform an installation on your system.
To specify destinations, consult the variables on top of the `Makefile` which can be overwritten by your environment.
This command will need extended permissions, so it might need to be executed by the `root` user, through `sudo` or a similar mechanism.
```bash
sudo make install
```

<!-- {% set from_source = True %} -->
<!-- {% include "02-Installation.md" %} -->
117 changes: 117 additions & 0 deletions doc/03-Configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Configuration

The configuration for Icinga Notifications is twofold.
The main configuration resides in the database,
shared between the Icinga Notifications daemon and Icinga Notifications Web.
However, as the Icinga Notifications daemon needs to know how to access this database and some further settings,
it needs its own configuration file as well.

This configuration is stored in `/etc/icinga-notifications/config.yml`.
See [config.example.yml](../config.example.yml) for an example configuration.

## Top Level Configuration

### HTTP API Configuration

The HTTP API listener can be used both for submission and for debugging purposes.

| Option | Description |
|----------------|----------------------------------------------------------------------|
| listen | Address to bind to, port included. (Example: `localhost:5680`) |
| debug-password | Password expected via HTTP Basic Authentication for debug endpoints. |

### Icinga Web 2

The `icingaweb2-url` is expected to point to the base directory of your Icinga Web 2 installation,
i.e., `https://example.com/icingaweb2/`, to be used for URL creation.

### Channel Plugin Directory

All available Icinga Notification Channel Plugins should resident in the `channel-plugin-dir`.
For a package install, this directory is `/usr/libexec/icinga-notifications/channels` and should not be altered.

## Database Configuration

Connection configuration for the database to which Icinga Notifications synchronizes monitoring data.
This is also the database used in Icinga Notifications Web to view and work with the data.

| Option | Description |
|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| type | **Optional.** Either `mysql` (default) or `pgsql`. |
| host | **Required.** Database host or absolute Unix socket path. |
| port | **Optional.** Database port. By default, the MySQL or PostgreSQL port, depending on the database type. |
| database | **Required.** Database name. |
| user | **Required.** Database username. |
| password | **Optional.** Database password. |
| tls | **Optional.** Whether to use TLS. |
| cert | **Optional.** Path to TLS client certificate. |
| key | **Optional.** Path to TLS private key. |
| ca | **Optional.** Path to TLS CA certificate. |
| insecure | **Optional.** Whether not to verify the peer. |
| options | **Optional.** List of low-level [database options](#database-options) that can be set to influence some Icinga Notifications internal default behaviours. |

### Database Options

Each of these configuration options are highly technical with thoroughly considered and tested default values that you
should only change when you exactly know what you are doing. You can use these options to influence the Icinga Notifications default
behaviour, how it interacts with databases, thus the defaults are usually sufficient for most users and do not need any
manual adjustments.

!!! important

Do not change the defaults if you do not have to!

| Option | Description |
|--------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| max_connections | **Optional.** Maximum number of database connections Icinga Notifications is allowed to open in parallel if necessary. Defaults to `16`. |
| max_connections_per_table | **Optional.** Maximum number of queries Icinga Notifications is allowed to execute on a single table concurrently. Defaults to `8`. |
| max_placeholders_per_statement | **Optional.** Maximum number of placeholders Icinga Notifications is allowed to use for a single SQL statement. Defaults to `8192`. |
| max_rows_per_transaction | **Optional.** Maximum number of rows Icinga Notifications is allowed to `SELECT`,`DELETE`,`UPDATE` or `INSERT` in a single transaction. Defaults to `8192`. |
| wsrep_sync_wait | **Optional.** Enforce [Galera cluster](#galera-cluster) nodes to perform strict cluster-wide causality checks. Defaults to `7`. |

## Logging Configuration

Configuration of the logging component used by Icinga Notifications.

| Option | Description |
|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| level | **Optional.** Specifies the default logging level. Can be set to `fatal`, `error`, `warn`, `info` or `debug`. Defaults to `info`. |
| output | **Optional.** Configures the logging output. Can be set to `console` (stderr) or `systemd-journald`. If not set, logs to systemd-journald when running under systemd, otherwise stderr. |
| interval | **Optional.** Interval for periodic logging defined as [duration string](#duration-string). Defaults to `"20s"`. |
| options | **Optional.** Map of component name to logging level in order to set a different logging level for each component instead of the default one. See [logging components](#logging-components) for details. |

### Logging Components

| Component | Description |
|-----------------|---------------------------------------------------------------------------|
| channel | Notification channels, their configuration and output. |
| database | Database connection status and queries. |
| icinga2 | Icinga 2 API communications, including the Event Stream. |
| incident | Incident management and changes. |
| listener | HTTP listener for event submission and debugging. |
| runtime-updates | Configuration changes through Icinga Notifications Web from the database. |

## Appendix

### Duration String

A duration string is a sequence of decimal numbers and a unit suffix, such as `"20s"`.
Valid units are `"ms"`, `"s"`, `"m"` and `"h"`.

### Galera Cluster

Icinga Notifications expects a more consistent behaviour from its database than a
[Galera cluster](https://mariadb.com/kb/en/what-is-mariadb-galera-cluster/) provides by default. To accommodate this,
Icinga Notifications sets the [wsrep_sync_wait](https://mariadb.com/kb/en/galera-cluster-system-variables/#wsrep_sync_wait) system
variable for all its database connections. Consequently, strict cluster-wide causality checks are enforced before
executing specific SQL queries, which are determined by the value set in the `wsrep_sync_wait` system variable.
By default, Icinga Notifications sets this to `7`, which includes `READ, UPDATE, DELETE, INSERT, REPLACE` query types and is
usually sufficient. Unfortunately, this also has the downside that every single Icinga Notifications query will be blocked until
the cluster nodes resynchronise their states after each executed query, and may result in degraded performance.

However, this does not necessarily have to be the case if, for instance, Icinga Notifications is only allowed to connect to a
single cluster node at a time. This is the case when a load balancer does not randomly route connections to all the
nodes evenly, but always to the same node until it fails, or if your database cluster nodes have a virtual IP address
fail over assigned. In such situations, you can set the `wsrep_sync_wait` system variable to `0` in the
`/etc/icinga-notifications/config.yml` file to disable it entirely, as Icinga Notifications doesn't have to wait for cluster
synchronisation then.
4 changes: 4 additions & 0 deletions doc/04-Upgrading.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Upgrading Icinga Notifications

Specific version upgrades are described below. Please note that version upgrades are incremental.
If you are upgrading across multiple versions, make sure to follow the steps for each of them.
3 changes: 3 additions & 0 deletions doc/05-Distributed-Setups.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Distributed Setups

To be done.
21 changes: 19 additions & 2 deletions doc/channel-plugin.md → doc/06-Channels.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,21 @@
## Channel Plugin
# Channels

After Icinga Notifications decides to send a notification of any kind, it will be passed to a channel plugin.
Such a channel plugin submits the notification event to a channel, e.g., email or a chat client.

Icinga Notifications comes packed with channel plugins, but also enables you to develop your own plugins.

To make those plugins available to Icinga Notifications, they must be placed in the [plugin channel directory](03-Configuration.md#channel-plugin-directory),
which is being done automatically for package installations.
Afterwards they can be configured through Icinga Notifications Web.

## Available Channels

- _email_: Email submission via SMTP
- _rocketchat_: Rocket.Chat
- _webhook_: Configurable HTTP/HTTPS queries for your backend

## Technical Channel Description

Channel plugins are processes that run continuously and independently of each other. They receive many requests over
their lifetime. They receive JSON-formatted requests on stdin and reply with JSON-formatted responses on stdout. The
Expand Down Expand Up @@ -73,4 +90,4 @@ Currently, the channel plugin include following three methods:
[(example)](../internal/channel/examples/get-info.json)
- `SendNotification`: Send the notifications. The `params` key should contain the information about the contact to be
notified, corresponding object, the incident and the triggered event.
[(example)](../internal/channel/examples/send-notification.json)
[(example)](../internal/channel/examples/send-notification.json)
Loading

0 comments on commit d0aaedc

Please sign in to comment.