[Elastic Agent] Fix composed.Verifier so files are removed when invalid

[blakerouse]

What does this PR do?

Fixes an issue in the composed.Verifier. Without this change if any verifier failed before the final verifier it would not remove the files and the last verifier in the chain would never be called to remove the files. This fixes the issue to always have each verifier have the ability to remove the files when it fails to verify.

Why is it important?

Without this change when a downloaded artifact gets corrupted, the Elastic Agent will keep trying to verify the same file, instead of removing it and re-downloading it from artifacts.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Closes [Elastic Agent] Composed Verifier does not always remove files #30280

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-02-08T21:01:17.309+0000

• Duration: 102 min 38 sec

Test stats 🧪

Test	Results
Failed	0
Passed	7232
Skipped	16
Total	7248

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[lykkin]

LGTM

[ph]

@blakerouse I would prefer we don't target 8.1/8.0, we are working to stabilize that release and we still don't know the other issue root cause. I find it risky to introduce it late.

[blakerouse]

I was able to test this with docker locally to confirm that it fixes the behavior:

1. First ran this once:

docker run -it -v $(pwd)/state:/state -e STATE_PATH=/state docker.elastic.co/beats/elastic-agent:8.2.0
# let it run for a bit
^c

2. Performed:

echo "baddata" >> state/data/downloads/filebeat-8.2.0-linux-x86_64.tar.gz

3. Ran it again:

docker run -it -v $(pwd)/state:/state -e STATE_PATH=/state docker.elastic.co/beats/elastic-agent:8.2.0

4. Resulted in the tar.gz and the .tar.gz.sha512 to be deleted and then it tried to download from the artifacts.

[blakerouse]

@ph I think this will mask the issue in 8.1.0, ultimately solving the corruption issue. But you are correct that it doesn't answer how it gets that way.

[mergify]

This pull request does not have a backport label. Could you fix it @blakerouse? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[blakerouse]

/package

[ph]

@blakerouse Yes, I think this is the problem we would mask the issues, and add traffic to the cloud deployment. :(

[ph]

Added backport to other version, lets see if mergify pick them up.