[Filebeat] Fix threatintel.indicator.url.full field not populating

[legoguy1000]

What does this PR do?

Properly sets the threatintel.indicator.url.full field for URL events for threatintel.abuseurl,misp,anomali,anomalithreatstream,otx

Why is it important?

url.full field wasn't being properly set.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

cd beats/x-pack/filebeat
TESTING_FILEBEAT_MODULES=threatintel mage -v pythonIntegTest

Related issues

Screenshots

Logs

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: P1llus commented: /test

• Start Time: 2021-06-29T12:21:36.503+0000

• Duration: 109 min 21 sec

• Commit: f02b12a

Test stats 🧪

Test	Results
Failed	0
Passed	7400
Skipped	1201
Total	8601

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	7400
Skipped	1201
Total	8601

[P1llus]

I think there is a few different ways to approach the same issue, but I would rather we depend more on the fields generated by the uri_parts processor. Instead of removing the remove_if_sucessfull setting, we should copy the value from url.original, so all the threatintel filesets should use uri_parts processor in combination with a set processor to copy_from the threatintel.indicator.url.original field. Do you think we could make that consistent over all the TI filesets?

- uri_parts:
    field: json.url
    target_field: threatintel.indicator.url
    keep_original: true
    remove_if_sucessful: true
    if: 'ctx.json.url != null'

- set:
    field: threatintel.indicator.url.full
    copy_from: threatintel.indicator.url.original
    ignore_empty_value: true

[P1llus]

/test

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)