Kitchen testing for Wazuh deployment with Puppet

.gitignore

@@ -25,3 +25,16 @@
 .project
 .envrc
 /inventory.yaml
+./kitchen/modules/.kitchen/logs/
+*.lock
+kitchen/.tmp/
+kitchen/.kitchen/
+kitchen/venv
+kitchen/*.xml
+kitchen/test/Dockerfile
+*.log
+*.pyc
+kitchen/.tmp/
+kitchen/.librarian/
+kitchen/.pytest_cache/
+kitchen/.*

kitchen/Gemfile

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+source "https://rubygems.org"
+
+# gem "rails"
+gem "test-kitchen"
+gem "kitchen-puppet"
+gem "kitchen-vagrant"
+gem 'kitchen-docker', '~> 2.3'
+gem "puppet"
+gem "librarian-puppet"

kitchen/Puppetfile

@@ -0,0 +1,26 @@
+#!/usr/bin/env ruby
+#^syntax detection
+
+forge "https://forgeapi.puppetlabs.com"
+
+# use dependencies defined in metadata.json
+#metadata
+
+mod "wazuh/wazuh"
+# use dependencies defined in Modulefile
+# modulefile
+
+# A module from the Puppet Forge
+# mod 'puppetlabs-stdlib'
+
+# A module from git
+# mod 'puppetlabs-ntp',
+#   :git => 'git://github.com/puppetlabs/puppetlabs-ntp.git'
+
+# A module from a git branch/tag
+# mod 'puppetlabs-apt',
+#   :git => 'https://github.com/puppetlabs/puppetlabs-apt.git',
+#   :ref => '1.4.x'
+
+# A module from Github pre-packaged tarball
+# mod 'puppetlabs-apache', '0.6.0', :github_tarball => 'puppetlabs/puppetlabs-apache'

kitchen/README.md

@@ -0,0 +1,176 @@
+**KITCHEN-PUPPET TESTING**
+
+**1.Building Kitchen Directory Structure**
+```
+├── chefignore
+├── Gemfile
+├── hieradata
+├── kitchen.yml
+├── manifests
+├── modules `should contain wazuh-puppet module`
+├── Puppetfile
+├── run.sh
+├── test
+```
+
+Find more details in the [official documentation](https://kitchen.ci/)
+
+**2. Required Gems**
+
+Kitchen basically works with `Ruby` gems and so, all required packages are available as gems. In our case, we would need the following gems to be installed. Found in the file `Gemfile` :
+
+```
+vagrant@master:~/wazuh-puppet/kitchen$ cat Gemfile
+# frozen_string_literal: true
+source "https://rubygems.org"
+
+# gem "rails"
+gem "test-kitchen"
+gem "kitchen-puppet"
+gem "kitchen-vagrant"
+gem 'kitchen-docker', '~> 2.3'
+gem "puppet"
+gem "librarian-puppet"
+```
+
+As we can see, we have gems for docker, vagrant, puppet, and kitchen itself.
+
+Once we have our list of gems prepared, we install them running the following command:
+
+```
+bundle install
+```
+
+**3. Kitchen Environment Configuration**
+
+In the file `kitchen.yml` we have to configure the machines were our tests will be running. This configuration includes information, such as : 
+* The virtualization tool `vagrant` or `docker`, 
+* The operating system image, 
+* Testing suites `testinfra` for example, etc ...
+
+- An initial example of `kitchen.yml` would be:
+
+```
+vagrant@master:~/wazuh-puppet/kitchen$ cat kitchen.yml
+---
+driver:
+  name: docker
+
+provisioner:
+  name: puppet_apply
+  manifests_path: manifests
+  modules_path: modules
+  hiera_data_path: hieradata
+
+platforms:
+  - name: ubuntu-manager_00
+    run_options: --ip 10.1.0.19
+    driver_config:
+      image: ubuntu:14.04
+      platform: ubuntu
+      hostname: manager00_ubuntu
+
+  - name: ubuntu-agent
+    driver_config:
+      image: ubuntu:14.04
+      platform: ubuntu
+      hostname: agent00_ubuntu
+
+suites:
+  - name: default
+    manifest: site.pp
+    verifier:
+      name: shell
+      command: py.test -v test/base
+```
+
+**4. Put Kitchen in action** 
+
+Once we have `kitchen.yml` prepared, then we can create the environment by running:
+
+```
+kitchen create
+```
+
+This way we will only have our machines created without installing the desired components to be tested. These components are represented by Wazuh stack components such as `wazuh-manager`, `wazuh-agent`, etc ...
+
+**5. Install the required components to be tested then**
+
+In `Puppet` case, to specify the `manifests` to be installed, we should configure the file 'manifests/site.pp', which by now it looks like:
+
+```
+node 'manager00_ubuntu' {
+  class { "wazuh::manager":
+        configure_wodle_openscap => false
+  }
+}
+node 'agent00_ubuntu' {
+  class { "wazuh::agent":
+        ossec_ip => "manager_ip",
+        configure_wodle_openscap => false
+  }
+}
+```
+
+As you can see, we only want to install `wazuh-manager` and `wazuh-agent`.
+
+Once `site.pp` is prepared, we run:
+```
+kitchen converge
+```
+
+**6. Testing time**
+
+`Kitchen` offers a large variety of testing types, such as:
+* Bats tests.
+* Serverspec tests.
+* Testinfra tests.
+* <Maybe there are more ' to be discovered later' >
+
+In our case, we think that `testinfra` is the best choice based on old experience. so and in order to implemente `testinfra` tests, we should indicate the testing suite command in `kitchen.yml` as indicated before:
+```
+suites:
+  - name: default
+    manifest: site.pp
+    verifier:
+      name: shell
+      command: py.test -v test/base
+```
+
+In the folder test/base, we put our tests. By now we implemented 2 tests, one for `wazuh-manager` and another one for `wazuh-agent`. Please check both here: 
+* [manager](https://github.com/wazuh/wazuh-puppet/blob/3.9.3_7.2.0_kitchen_tests/kitchen/test/base/test_wazuh_manager.py)
+* [agent](https://github.com/wazuh/wazuh-puppet/blob/3.9.3_7.2.0_kitchen_tests/kitchen/test/base/test_wazuh_agent.py)
+
+Once we have our suite prepared, then we run:
+
+```
+kitchen verify
+```
+
+And in a successful testing attempt we can get something like:
+
+```
+-----> Starting Kitchen (v2.2.5)
+-----> Verifying <default-ubuntu-manager-00>...
+       [Shell] Verify on instance default-ubuntu-manager-00  ...
+
+============================= test session starts ==============================
+platform linux -- Python 3.4.3, pytest-4.6.4, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python3.4
+cachedir: .pytest_cache
+rootdir: /home/vagrant/wazuh-puppet/kitchen
+plugins: testinfra-3.0.5
+collecting ... collected 8 items
+
+test/base/test_wazuh_agent.py::test_wazuh_agent_package SKIPPED          [ 12%]
+test/base/test_wazuh_agent.py::test_wazuh_processes_running[ossec-agentd-ossec] SKIPPED [ 25%]
+test/base/test_wazuh_agent.py::test_wazuh_processes_running[ossec-execd-root] SKIPPED [ 37%]
+test/base/test_wazuh_agent.py::test_wazuh_processes_running[ossec-syscheckd-root] SKIPPED [ 50%]
+test/base/test_wazuh_agent.py::test_wazuh_processes_running[wazuh-modulesd-root] SKIPPED [ 62%]
+test/base/test_wazuh_manager.py::test_wazuh_agent_package PASSED         [ 75%]
+test/base/test_wazuh_manager.py::test_wazuh_packages_are_installed PASSED [ 87%]
+test/base/test_wazuh_manager.py::test_wazuh_services_are_running PASSED  [100%]
+
+===================== 3 passed, 5 skipped in 1.18 seconds ======================
+       Finished verifying <default-ubuntu-manager-00> (0m2.16s).
+-----> Kitchen is finished. (0m4.51s)
+```

kitchen/chefignore

@@ -0,0 +1 @@
+.kitchen

kitchen/kitchen.yml

@@ -0,0 +1,52 @@
+---
+driver:
+  name: docker
+  privileged: true
+  use_sudo: false
+
+provisioner:
+  name: puppet_apply
+  manifests_path: manifests
+  modules_path: modules
+  hiera_data_path: hieradata
+
+platforms:
+  - name: ubuntu-manager_00
+    driver_config:
+      image: ubuntu:14.04
+      platform: ubuntu
+      hostname: manager00_ubuntu
+
+  - name: ubuntu-agent
+    driver_config:
+      image: ubuntu:14.04
+      platform: ubuntu
+      hostname: agent00_ubuntu
+
+  - name: centos-manager_00
+    driver_config:
+      image: centos:7
+      platform: centos
+      hostname: manager00_centos
+      run_command: /usr/sbin/init
+      dockerfile: test/Dockerfile
+      build_options:
+        rm: true
+
+  - name: centos-agent
+    driver_config:
+      image: centos:7
+      platform: centos
+      hostname: agent00_centos
+      run_command: /usr/sbin/init
+      run_command: /usr/lib/systemd/systemd
+      dockerfile: test/Dockerfile
+      build_options:
+        rm: true
+
+suites:
+  - name: default
+    manifest: site.pp
+    verifier:
+      name: shell
+      command: py.test -v test/base

kitchen/manifests/site.pp

@@ -0,0 +1,22 @@
+node 'manager00_ubuntu' {
+  class { "wazuh::manager":
+        configure_wodle_openscap => false
+  }
+}
+node 'agent00_ubuntu' {
+  class { "wazuh::agent":
+        ossec_ip => "10.1.0.1",
+        configure_wodle_openscap => false
+  }
+}
+node 'manager00_centos' {
+  class { "wazuh::manager":
+        configure_wodle_openscap => true
+  }
+}
+node 'agent00_centos' {
+  class { "wazuh::agent":
+        ossec_ip => "10.1.0.3",
+        configure_wodle_openscap => true
+  }
+}

kitchen/manifests/site.pp.template

@@ -0,0 +1,22 @@
+node 'manager00_ubuntu' {
+  class { "wazuh::manager":
+        configure_wodle_openscap => false
+  }
+}
+node 'agent00_ubuntu' {
+  class { "wazuh::agent":
+        ossec_ip => "ubuntu_manager_ip",
+        configure_wodle_openscap => false
+  }
+}
+node 'manager00_centos' {
+  class { "wazuh::manager":
+        configure_wodle_openscap => true
+  }
+}
+node 'agent00_centos' {
+  class { "wazuh::agent":
+        ossec_ip => "centos_manager_ip",
+        configure_wodle_openscap => true
+  }
+}

kitchen/run.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+rm -rf .kitchen/logs/*
+rm -rf .kitchen/def*
+rm -rf ./manifests/se*
+kitchen destroy all
+docker rm -f $(docker ps -aq)
+kitchen create
+ubuntu_manager_ip="$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' `docker ps | awk '{print $NF}' | grep  ubuntu | grep manager`)"
+centos_manager_ip="$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' `docker ps | awk '{print $NF}' | grep  centos | grep manager`)"
+cp ./manifests/site.pp.template ./manifests/site.pp
+sed -i 's/ubuntu_manager_ip/'${ubuntu_manager_ip}'/g' ./manifests/site.pp
+sed -i 's/centos_manager_ip/'${centos_manager_ip}'/g' ./manifests/site.pp
+kitchen converge
+kitchen verify

kitchen/test/Dockerfile.template

@@ -0,0 +1,20 @@
+FROM centos:7
+ENV container docker
+RUN yum clean all
+RUN yum install -y sudo openssh-server openssh-clients which curl
+RUN [ -f "/etc/ssh/ssh_host_rsa_key" ] || ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
+RUN [ -f "/etc/ssh/ssh_host_dsa_key" ] || ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key -N ''
+RUN if ! getent passwd kitchen; then                 useradd -d /home/kitchen -m -s /bin/bash -p '*' kitchen;               fi
+RUN echo "kitchen ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
+RUN echo "Defaults !requiretty" >> /etc/sudoers
+RUN mkdir -p /home/kitchen/.ssh
+RUN chown -R kitchen /home/kitchen/.ssh
+RUN chmod 0700 /home/kitchen/.ssh
+RUN touch /home/kitchen/.ssh/authorized_keys
+RUN chown kitchen /home/kitchen/.ssh/authorized_keys
+RUN chmod 0600 /home/kitchen/.ssh/authorized_keys
+RUN mkdir -p /run/sshd
+RUN echo <YOUR PUBLIC KEY> kitchen_docker_key >> /home/kitchen/.ssh/authorized_keys
+EXPOSE 1515/tcp
+EXPOSE 1515/udp
+RUN yum install -y openssl

kitchen/test/base/test_wazuh_agent.py

@@ -0,0 +1,28 @@
+import functools
+import os
+import pytest
+import testinfra
+
+test_host = testinfra.get_host('paramiko://{KITCHEN_USERNAME}@{KITCHEN_HOSTNAME}:{KITCHEN_PORT}'.format(**os.environ), ssh_identity_file=os.environ.get('KITCHEN_SSH_KEY'))
+
+@pytest.mark.filterwarnings('ignore')
+@pytest.mark.skipif('manager' in os.environ.get('KITCHEN_INSTANCE'), reason='Skip on wazuh manager instances')
+def test_wazuh_agent_package(host):
+    name = "wazuh-agent"
+    version = "3.9.3"
+    pkg = host.package(name)
+    assert pkg.is_installed
+    assert pkg.version.startswith(version)
+
+
+@pytest.mark.filterwarnings('ignore')
+@pytest.mark.skipif('manager' in os.environ.get('KITCHEN_INSTANCE'), reason='Skip on wazuh manager instances')
+@pytest.mark.parametrize("wazuh_service, wazuh_owner", (
+        ("ossec-agentd", "ossec"),
+        ("ossec-execd", "root"),
+        ("ossec-syscheckd", "root"),
+        ("wazuh-modulesd", "root"),
+))
+def test_wazuh_processes_running(host, wazuh_service, wazuh_owner):
+    master = host.process.get(user=wazuh_owner, comm=wazuh_service)
+    assert master.args == "/var/ossec/bin/" + wazuh_service