KeyError: 'type' when processing challenge error response.

[CompPhy]

SUMMARY

We are getting invalid responses from ZeroSSL challenge requests, but can't tell why they are invalid. Instead, the crypto module is crashing with the below exception when trying to handle the challenge response. It looks like there just needs to be a check on type attribute here to make sure it actually exists. Maybe just print out the response body if there's not a better way to decode the error here?

Please note, this set of tasks has worked previously. This request is renewing an existing certificate, which might be related to the error. However, we can't even see the error to be sure because of this exception.

"module_stderr": "Traceback (most recent call last):
File "", line 100, in
File "", line 92, in _ansiballz_main
File "", line 41, in invoke_module
File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_certificate.py", line 950, in
File "/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_certificate.py", line 920, in main
File "/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_certificate.py", line 766, in finish_challenges
File "/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/challenges.py", line 316, in wait_for_validation
File "/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/challenges.py", line 234, in raise_error
File "/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/errors.py", line 30, in format_error_problem
KeyError: 'type'

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.crypto.acme_certificate

ANSIBLE VERSION

ansible [core 2.11.12] 
  config file = None
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.16 (default, Jan 17 2023, 18:50:05) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)]
  jinja version = 2.11.3
  libyaml = True

COLLECTION VERSION

# /usr/local/lib/python3.8/site-packages/ansible_collections
Collection       Version
---------------- -------
community.crypto 1.9.8  

CONFIGURATION

[defaults]
hash_behaviour = merge
inventory = inventory
timeout = 120
host_key_checking = False
callbacks_enabled = profile_tasks, profile_roles
vault_password_file = <REDACTED>

[connection]
pipelining = true

[ssh_connection]
ssh_args = "-C -o ControlMaster=auto -o ControlPersist=600s"

[galaxy]
server_list = automation_hub, release_galaxy

[galaxy_server.automation_hub]
<REDACTED>

[galaxy_server.release_galaxy]
<REDACTED>

OS / ENVIRONMENT

Execution Environment based on quay.io/ansible/ansible-runner:latest

STEPS TO REPRODUCE

- name: "Generate the private key file for the SSL certificate."
  community.crypto.openssl_privatekey:
    path: "{{ zerossl_key }}"
    mode: 0644

- name: "Generate a Certificate Signing Request for {{ zerossl_common_name }}."
  community.crypto.openssl_csr:
    path: "{{ zerossl_csr }}"
    privatekey_path: "{{ zerossl_key }}"
    common_name: "{{ zerossl_common_name }}"
    organization_name: <REDACTED>

- name: "Make sure the account key is setup on the Vargo account."
  community.crypto.acme_account:
    account_key_src: "{{ zerossl_account_key }}"
    external_account_binding:
      <REDACTED>
    contact:
      <REDACTED>
    terms_agreed: true
    acme_directory: https://acme.zerossl.com/v2/DV90
    acme_version: 2
    request_timeout: 60
    state: present

- name: "Generate the ACME challenge for {{ zerossl_common_name }}, using DNS method."
  community.crypto.acme_certificate:
    account_key_src: "{{ zerossl_account_key }}"
    account_email: "{{ zerossl_account_email }}"
    csr: "{{ zerossl_csr }}"
    cert: "{{ zerossl_crt }}"
    chain: "{{ zerossl_intermediate }}"
    fullchain: "{{ zerossl_fullchain }}"
    challenge: dns-01
    acme_directory: https://acme.zerossl.com/v2/DV90
    acme_version: 2
    request_timeout: 60
  register: zerossl_challenge

- name: Set record facts.
  set_fact:
    the_record_name: "{{ zerossl_challenge.challenge_data[zerossl_common_name]['dns-01'].record }}."
    the_record_data: "{{ zerossl_challenge.challenge_data[zerossl_common_name]['dns-01'].resource_value | regex_replace('^(.*)$', '\"\\1\"') }}"

- name: "Fulfill the DNS challenge via GoDaddy API."
  include_role:
    name: djungle_io.godaddy_ansible_role
  vars:
    godaddy_domain_name: "{{ zerossl_domain }}"
    godaddy_record_name: "{{ the_record_name }}"
    record_data: "{{ the_record_data }}"
    type: TXT
    ttl: 600
  when: zerossl_challenge is changed and zerossl_common_name in zerossl_challenge.challenge_data

- name: Make sure DNS is resolving before completing the challenge.
  debug:
    msg: "Waiting for {{ the_record_name }} to resolve in DNS."
  until: lookup('community.general.dig', the_record_name, qtype='TXT') == the_record_data
  retries: 30
  delay: 10

# EXCEPTION HAPPENS ON THIS TASK
- name: "Let the challenge be validated and retrieve the certificate."
  community.crypto.acme_certificate:
    data: "{{ zerossl_challenge }}"
    account_key_src: "{{ zerossl_account_key }}"
    account_email: "{{ zerossl_account_email }}"
    csr: "{{ zerossl_csr }}"
    cert: "{{ zerossl_crt }}"
    chain: "{{ zerossl_intermediate }}"
    fullchain: "{{ zerossl_fullchain }}"
    challenge: dns-01
    acme_directory: https://acme.zerossl.com/v2/DV90
    acme_version: 2
    request_timeout: 60
  when: zerossl_challenge is changed

EXPECTED RESULTS

We should get some information about the error response here, not a traceback from the module crashing.

ACTUAL RESULTS

Module crashed with this traceback.

  "module_stderr": "Traceback (most recent call last):
  File \"<stdin>\", line 100, in <module>
  File \"<stdin>\", line 92, in _ansiballz_main
  File \"<stdin>\", line 41, in invoke_module
  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code
    exec(code, run_globals)
  File \"/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_certificate.py\", line 950, in <module>
  File \"/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_certificate.py\", line 920, in main
  File \"/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_certificate.py\", line 766, in finish_challenges
  File \"/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/challenges.py\", line 316, in wait_for_validation
  File \"/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/challenges.py\", line 234, in raise_error
  File \"/tmp/ansible_community.crypto.acme_certificate_payload_2zt5rlrh/ansible_community.crypto.acme_certificate_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/errors.py\", line 30, in format_error_problem
KeyError: 'type'

[felixfontein]

There seems to be a bug in the error handling code, so you only see a crash of the error handling code instead of the actual error returned by the CA. I'll take a look at this later today...

[felixfontein]

(Generally the module works with ZeroSSL, I renewed some test certs with them during the weekend and no modification was done to the code since then.)

[felixfontein]

#652 should fix this.