Allow customization of netty channel handles before and during decompression

[cwperks]

Description

Companion PR: opensearch-project/security#3418

In some cases, a request is considered invalid from its headers and its unnecessary to inspect the body of the request. For compressed requests, decompressed bodies can sometimes be quite large so its a benefit to skip decompression if the body is only being discarded.

This PR allows a plugin to extend the pipeline and determine whether decompression can be skipped in the pipeline if the step is unnecessary.

Related Issues

• Related [Feature] Improve compressed request handling #10260

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26412/
• CommitID: 0925b7e
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Compatibility status:

Checks if related components are compatible with change ddaca29

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git]

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26414/
• CommitID: a8d9b73
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26417/
• CommitID: 5188697
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26418/
• CommitID: cd9e72f
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26419/
• CommitID: bf2d707
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26420/
• CommitID: bfc3ceb
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/26424/
• CommitID: 2584e94

[codecov]

Codecov Report

Merging #10261 (ddaca29) into main (2965e69) will increase coverage by 0.04%.
Report is 16 commits behind head on main.
The diff coverage is 79.05%.

@@             Coverage Diff              @@
##               main   #10261      +/-   ##
============================================
+ Coverage     71.16%   71.21%   +0.04%     
- Complexity    58297    58397     +100     
============================================
  Files          4830     4843      +13     
  Lines        274903   275193     +290     
  Branches      40059    40068       +9     
============================================
+ Hits         195637   195980     +343     
+ Misses        62882    62814      -68     
- Partials      16384    16399      +15     

Files	Coverage Δ
...arch/telemetry/metrics/DefaultMetricsRegistry.java	100.00% <100.00%> (ø)
...ch/telemetry/metrics/noop/NoopMetricsRegistry.java	100.00% <100.00%> (ø)
...va/org/opensearch/telemetry/metrics/tags/Tags.java	100.00% <100.00%> (ø)
...org/opensearch/repositories/url/URLRepository.java	72.00% <100.00%> (ø)
...g/opensearch/transport/netty4/Netty4Transport.java	73.65% <100.00%> (ø)
...opensearch/repositories/azure/AzureRepository.java	68.88% <100.00%> (-1.33%)	⬇️
...g/opensearch/repositories/hdfs/HdfsRepository.java	61.90% <100.00%> (ø)
...java/org/opensearch/repositories/s3/S3Service.java	75.00% <100.00%> (ø)
.../org/opensearch/telemetry/OTelTelemetryPlugin.java	100.00% <100.00%> (ø)
...rg/opensearch/telemetry/OTelTelemetrySettings.java	100.00% <100.00%> (ø)
... and 40 more

... and 472 files with indirect coverage changes

[peternied]

Thanks for getting this out @cwperks

I'm seeing a lot of changes - whereas I'd prefer we slim this down to only the createHeaderVerifier. It looks like there were many unrelated changes that had to be propped up to support generating requests objects in a way they are usable by the existing RestHandler wrapper which reuse many aspects of the SecurityPlugins current logic.

I'm going to spend some cycles wrapping my head around this change and come up with some prototypes for other suggestions. This is definitely on the right path - maybe we can figure out how to adjust the scope of changes.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26454/
• CommitID: 0a0214d
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL:
• CommitID: 104c512
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/26459/
• CommitID: 001b7cf
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/27184/
• CommitID: ddaca29
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/27198/
• CommitID: ddaca29
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[peternied]

Nicely done, thanks for the test coverage and the enhancement to the request pipeline.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/27209/
• CommitID: ddaca29
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/27220/
• CommitID: ddaca29

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-10261-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 dad525aefaab01b8452f9db8d7fba70a4d3b5cc8
# Push it to GitHub
git push --set-upstream origin backport/backport-10261-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-10261-to-2.x.

[opensearch-trigger-bot]

The backport to 2.11 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.11 2.11
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.11
# Create a new branch
git switch --create backport/backport-10261-to-2.11
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 dad525aefaab01b8452f9db8d7fba70a4d3b5cc8
# Push it to GitHub
git push --set-upstream origin backport/backport-10261-to-2.11
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.11

Then, create a pull request where the base branch is 2.11 and the compare/head branch is backport/backport-10261-to-2.11.