[BUG] update_attributes on folder with network scan enabled (and a network scan result present) fails

[Mik3yZ]

Describe the bug
When a network scan is defined and a result is there for a folder, we can't update the folder attributes.

failed: [RemoteTest1 -> localhost] (item={'name': 'Graveyard', 'path': 'graveyard', 'contactgroups': ['g_cg_windowsengineers', 'g_cg_linuxengineers', 'g_cg_vmwareengineers', 'g_cg_hypervengineers', 'g_cg_networkengineers']}) => {"ansible_loop_var": "item", "changed": false, "item": {"contactgroups": ["g_cg_windowsengineers", "g_cg_linuxengineers", "g_cg_vmwareengineers", "g_cg_hypervengineers", "g_cg_networkengineers"], "name": "Graveyard", "path": "graveyard"}, "msg": "Error calling API. HTTP code 400. Details: b'{\"title\": \"Bad Request\", \"status\": 400, \"detail\": \"These fields have problems: update_attributes\", \"fields\": {\"update_attributes\": {\"network_scan_result\": [\"Not a string, but a dict\", \"Tag group name must start with \\'tag_\\'\", \"Unknown field.\"]}}}', "}

Component Name
Component Name: folder_module

Ansible Version

$ ansible --version
ansible [core 2.14.13]
  config file = /home/<uid>/repos/ansible-checkmk/ansible.cfg
  configured module search path = ['/home/<uid>/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/<uid>/venvs/ansible-checkmk/lib64/python3.9/site-packages/ansible
  ansible collection location = /home/<uid>/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/<uid>/venvs/ansible-checkmk/bin/ansible
  python version = 3.9.16 (main, May 29 2023, 00:00:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4)] (/home/<uid>/venvs/ansible-checkmk/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Checkmk Version and Edition

Checkmk Enterprise Edition 2.2.0p17

Collection Version

$ ansible-galaxy collection list
...
checkmk.general        4.0.0
...

To Reproduce

We create our folder structure via a var file:

---
folders:
  - { name: "Graveyard", path: "graveyard", contactgroups: [ "g_cg_windowsengineers", "g_cg_linuxengineers", "g_cg_vmwareengineers", "g_cg_hypervengineers", "g_cg_networkengineers" ] }

And we then create the folders with a loop.

---
- name: Include vars of folders.yml
  ansible.builtin.include_vars:
    file: vars/folders.yml

- name: "Create folders"
  checkmk.general.folder:
    server_url: "{{ master_site_baseurl }}"
    site: "{{ master_site }}"
    automation_user: "{{ automation_user }}"
    automation_secret: "{{ automation_secret }}"
    path: "{{ folder_basepath }}/{{ item.path.lower() }}"
    name: "{{ item.name }}"
    update_attributes:
      labels: "{{ item.labels|default(omit) }}"
    state: "{{ item.state | default('present') }}"
  delegate_to: localhost
  loop: "{{ folders }}"

- name: "Update folder contactgroups"
  checkmk.general.folder:
    server_url: "{{ master_site_baseurl }}"
    site: "{{ master_site }}"
    automation_user: "{{ automation_user }}"
    automation_secret: "{{ automation_secret }}"
    path: "{{ folder_basepath }}/{{ item.path.lower() }}"
    name: "{{ item.name }}"
    update_attributes:
      contactgroups:
        groups: "{{ item.contactgroups }}"
        use: true
        use_for_services: true
        recurse_use: true
        recurse_perms: true
    state: "{{ item.state | default('present') }}"
  delegate_to: localhost
  loop: "{{ folders }}"
  when: item.contactgroups is defined

The first time this runs succesfully, but when we define a network scan on this folder (and wait for it to finish so a network scan result is there) and run the playbook we receive the failure on the "Update folder contactgroups" play:

failed: [RemoteTest1 -> localhost] (item={'name': 'Graveyard', 'path': 'graveyard', 'contactgroups': ['g_cg_windowsengineers', 'g_cg_linuxengineers', 'g_cg_vmwareengineers', 'g_cg_hypervengineers', 'g_cg_networkengineers']}) => {"ansible_loop_var": "item", "changed": false, "item": {"contactgroups": ["g_cg_windowsengineers", "g_cg_linuxengineers", "g_cg_vmwareengineers", "g_cg_hypervengineers", "g_cg_networkengineers"], "name": "Graveyard", "path": "graveyard"}, "msg": "Error calling API. HTTP code 400. Details: b'{\"title\": \"Bad Request\", \"status\": 400, \"detail\": \"These fields have problems: update_attributes\", \"fields\": {\"update_attributes\": {\"network_scan_result\": [\"Not a string, but a dict\", \"Tag group name must start with \\'tag_\\'\", \"Unknown field.\"]}}}', "}

Expected behavior
We expect the folder to just be updated

Actual behavior
Error message

[Mik3yZ]

Small addition:

if we remove the network scan alone and try it again it still fails. If we remove the network_scan_result also (from the .wato file of the folder) it succeeds.

[robin-checkmk]

Hi @Mik3yZ, sorry for the late answer, and thanks for reporting this. Did you test this with the REST API directly as well? I have a feeling, that this is an upstream issue. If you could test that quickly, that'd be great, then I can report it internally.

[Mik3yZ]

Hi Robin,

If we execute this via the API it just works. I Checked the code of the folders module, and what it does it that it fetches all the folder attributes with the function get_current_folder_state. This function is then called in run_module and it does a dict_merge with the update_attributes values.

Somewhere in here, i would say, we should exclude the network_scan_results

[lgetwan]

Hi @Mik3yZ,

I recently rewrote the folder module (see #536), but it's not yet merged into the devel branch.
Thanks for reporting that issue, so I was able to change that before it's merged. :-)
I already considered the "meta_data" attributes to be dropped, but now I also dropped the "network_scan_results".
You can checkout my feature branch for testing, if you like.

Best regards
Lars

[robin-checkmk]

This was actually implemented with #558.

[Dennis-Q]

@robin-checkmk it seems this issue isn't really resolved. I'm running the latest collection (5.1.0) and still get this error:

failed: [Remote -> localhost] (item={'name': 'Graveyard', 'path': 'graveyard', 'contactgroups': ['g_cg_windowsengineers', 'g_cg_linuxengineers', 'g_cg_vmwareengineers', 'g_cg_hypervengineers', 'g_cg_networkengineers'], 'contact': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"contact": false, "contactgroups": ["g_cg_windowsengineers", "g_cg_linuxengineers", "g_cg_vmwareengineers", "g_cg_hypervengineers", "g_cg_networkengineers"], "name": "Graveyard", "path": "graveyard"}, "msg": "400 - Bad request: Parameter or validation failure Details: b'{\"title\": \"Bad Request\", \"status\": 400, \"detail\": \"These fields have problems: update_attributes\", \"fields\": {\"update_attributes\": {\"network_scan_result\": [\"Not a string, but a dict\", \"Tag group name must start with \\'tag_\\'\", \"Unknown field.\"]}}}'"}

Ansible-collection list output:

$ ansible-galaxy collection list
/home/<user>/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
ansible.posix                            1.5.4  
ansible.utils                            5.0.0  
brocade.fos                              1.3.3  
checkmk.general                          5.1.0 

To me, it looks like the issue isn't really fixed in the latest releases. By the looks of it, the update-module still takes the 'network_scan_result', which fails since I think it should be ommited as it's read-only (should be, although it reports about incorrect type).

Easy steps to recreate:

• Enable network scan on an existing folder
• Wait for the scan to happen (check properties of folder --> Last Scan Result should not be 'No scan performed yet. '
• Use 'checkmk.general.folder' module to update folder using 'update_attributes' option.

More details can be found in OP of Mik3yZ.

Can we re-open this issue or should I create a new issue for it?

[robin-checkmk]

@lgetwan can you take a gander at this, when you find the time? Thanks!

[lgetwan]

Fixed in #648 (which is under review).
Was just a typo...