Delete API for weighted round robin search routing #4400

anshu1106 · 2022-09-03T16:28:39Z

Description

This pull request adds DELTE api to delete weights for weighted routing policy. This in turn disable the deature.

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2022-09-03T16:48:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2599/
CommitID: 8ce5255

server/src/internalClusterTest/java/org/opensearch/search/SearchWRRIT.java

gbbafna · 2022-09-05T06:55:03Z

server/src/internalClusterTest/java/org/opensearch/search/SearchWRRIT.java

+        }
+
+    }
+


should we remove the weights and then assert again that requests are equally distributed ?

Sure, any pointers on which stat I can assert on for request distribution?

sorry missed this. You can look for node/stats API call for this.

gbbafna · 2022-09-05T07:35:03Z

...org/opensearch/action/admin/cluster/shards/routing/wrr/get/TransportGetWRRWeightsAction.java

+ *
+ * @opensearch.internal
+ */
+public class TransportGetWRRWeightsAction extends TransportClusterManagerNodeReadAction<


should we extend HandledTransportAction instead of TransportClusterManagerNodeReadAction ? I do see that TransportClusterManagerNodeReadAction are used for GET actions which needs cluster state . but just thinking about the benefits/cost for the same .

...org/opensearch/action/admin/cluster/shards/routing/wrr/get/TransportGetWRRWeightsAction.java

server/src/main/java/org/opensearch/client/ClusterAdminClient.java

gbbafna

Delete API LGTM.

github-actions · 2022-09-07T05:56:23Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2798/
CommitID: c6ed81c

github-actions · 2022-09-20T13:06:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3243/
CommitID: 7b3e758

github-actions · 2022-09-22T08:11:59Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3312/
CommitID: 3ce3c22

github-actions · 2022-09-27T09:54:12Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3464/
CommitID: a32ae28

github-actions · 2022-09-27T09:58:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3465/
CommitID: 6eb285f

github-actions · 2022-09-27T10:31:07Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3466/
CommitID: f5514c4

github-actions · 2022-09-27T17:23:48Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3481/
CommitID: 7b37962

github-actions · 2022-09-27T17:47:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3483/
CommitID: 7a8a4d2

github-actions · 2022-09-28T13:42:19Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3527/
CommitID: b7d9580

github-actions · 2022-10-06T08:26:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3914/
CommitID: e5c4c5a

github-actions · 2022-10-06T10:27:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3921/
CommitID: 91d7759

codecov-commenter · 2022-10-06T10:29:41Z

Codecov Report

Merging #4400 (6a4c8d5) into main (d15795a) will increase coverage by 0.12%.
The diff coverage is 44.81%.

@@             Coverage Diff              @@
##               main    #4400      +/-   ##
============================================
+ Coverage     70.66%   70.78%   +0.12%     
- Complexity    57578    57698     +120     
============================================
  Files          4661     4675      +14     
  Lines        276662   276918     +256     
  Branches      40325    40347      +22     
============================================
+ Hits         195501   196016     +515     
+ Misses        64926    64684     -242     
+ Partials      16235    16218      -17

Impacted Files	Coverage Δ
.../delete/DeleteDecommissionStateRequestBuilder.java	`0.00% <0.00%> (ø)`
...te/ClusterDeleteWeightedRoutingRequestBuilder.java	`0.00% <0.00%> (ø)`
...apshots/restore/RestoreSnapshotRequestBuilder.java	`15.78% <0.00%> (-0.88%)`	⬇️
.../org/opensearch/client/support/AbstractClient.java	`32.30% <0.00%> (-0.63%)`	⬇️
.../java/org/opensearch/common/util/FeatureFlags.java	`50.00% <ø> (ø)`
...ateway/TransportNodesListGatewayStartedShards.java	`51.00% <0.00%> (-0.68%)`	⬇️
...h/index/shard/RemoveCorruptedShardDataCommand.java	`84.87% <ø> (+3.36%)`	⬆️
...java/org/opensearch/index/shard/StoreRecovery.java	`67.88% <ø> (+0.01%)`	⬆️
...h/index/store/InMemoryRemoteSnapshotDirectory.java	`0.00% <0.00%> (ø)`
...in/java/org/opensearch/indices/IndicesService.java	`69.65% <0.00%> (-0.25%)`	⬇️
... and 482 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

github-actions · 2022-10-06T11:51:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3926/
CommitID: 6554f3a

github-actions · 2022-10-06T12:03:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3927/
CommitID: e864111

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

imRishN · 2022-10-06T12:39:10Z

...ction/admin/cluster/shards/routing/weighted/delete/TransportDeleteWeightedRoutingAction.java

+            ActionListener.delegateFailure(
+                listener,
+                (delegatedListener, response) -> {
+                    delegatedListener.onResponse(new ClusterDeleteWeightedRoutingResponse(response.isAcknowledged()));


Why do we need to delegate this? The service method definition can be changed to ClusterDeleteWRResponse?

Removed delegatedListener

imRishN · 2022-10-06T12:40:56Z

server/src/main/java/org/opensearch/cluster/routing/WeightedRoutingService.java

+                logger.info("Deleting weighted routing metadata from the cluster state");
+                Metadata.Builder mdBuilder = Metadata.builder(currentState.metadata());
+                mdBuilder.removeCustom(WeightedRoutingMetadata.TYPE);
+                return ClusterState.builder(currentState).metadata(mdBuilder).build();


If we just wipe off the metadata will it internally handle to use ARS strategy? Or what will happen to subsequent requests?

Yeah, subsequent requests will use ARS or attribute based routing. For multi-az clusters, currently ARS is not supported.

imRishN · 2022-10-06T12:42:12Z

server/src/main/java/org/opensearch/cluster/routing/WeightedRoutingService.java

+
+            @Override
+            public void onFailure(String source, Exception e) {
+                logger.warn(() -> new ParameterizedMessage("failed to remove weighted routing metadata from cluster state [{}]", e));


nit: the msg looks like is exoecting a cluster state but provided an exception.
We can make this logger.error and pass the exception in it

github-actions · 2022-10-06T12:55:24Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3939/
CommitID: 56fe6af

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2022-10-06T13:25:49Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3952/
CommitID: e8cda62

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2022-10-06T16:08:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3966/
CommitID: a4fe992

imRishN · 2022-10-06T16:52:35Z

server/src/main/java/org/opensearch/cluster/routing/WeightedRoutingService.java

+                logger.error(
+                    () -> new ParameterizedMessage(
+                        "failed to remove weighted routing metadata from cluster state with an exception [{}]",
+                        e
+                    )
+                );


if there's no other parameter we can keep it like - logger.error(msg, e)

github-actions · 2022-10-06T17:43:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3986/
CommitID: 5321b95

github-actions · 2022-10-06T18:13:20Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3993/
CommitID: 6a1daad

anshu1106 · 2022-10-06T18:17:39Z

Seeing unrelated test failures

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2022-10-07T05:24:19Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/4092/
CommitID: db8283c

github-actions · 2022-10-07T05:40:25Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/4095/
CommitID: b6ed9e5

imRishN

Thanks @anshu1106 for the changes

Bukhtawar · 2022-10-12T07:20:37Z

...in/java/org/opensearch/rest/action/admin/cluster/RestClusterDeleteWeightedRoutingAction.java

+    @Override
+    public List<Route> routes() {
+        return singletonList(new Route(DELETE, "/_cluster/routing/awareness/weights"));
+    }


do we want to support explicit attribute deletion?

yeah, but this will require few changes in the way data is persisted in cluster state as well. Create an issue to work on this #4747

github-actions · 2022-10-13T13:27:02Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/4576/
CommitID: 6a4c8d5

…t#4400) * Delete API for weighted round robin search routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Weighted round-robin scheduling policy for shard coordination traffic… (#4241) * Add PUT api to update shard routing weights (#4272) * Add GET api to get shard routing weights (#4275) * Fix weighted routing metadata deserialization error during node restart (#4691) * Delete API for weighted round robin search routing (#4400) * Mark apis experimental Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

…t#4400) * Delete API for weighted round robin search routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 requested review from a team and reta as code owners September 3, 2022 16:28

gbbafna reviewed Sep 5, 2022

View reviewed changes

anshu1106 mentioned this pull request Sep 7, 2022

Weighted round-robin scheduling policy for shard coordination traffic… #4241

Merged

5 tasks

anshu1106 force-pushed the feature/wrr-delete-api branch from 7b37962 to 7a8a4d2 Compare September 27, 2022 17:21

anshu1106 mentioned this pull request Oct 6, 2022

[META] Weighted Routing Issue List #4526

Open

20 tasks

anshu1106 force-pushed the feature/wrr-delete-api branch from 6554f3a to e864111 Compare October 6, 2022 11:38

Squash all commits in one

56fe6af

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 force-pushed the feature/wrr-delete-api branch from e864111 to 56fe6af Compare October 6, 2022 12:27

imRishN suggested changes Oct 6, 2022

View reviewed changes

Address review comments

e8cda62

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Fix build error

a4fe992

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

imRishN reviewed Oct 6, 2022

View reviewed changes

anshu1106 force-pushed the feature/wrr-delete-api branch from 5321b95 to 6a1daad Compare October 6, 2022 17:49

Address review comment

db8283c

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 force-pushed the feature/wrr-delete-api branch from 6a1daad to db8283c Compare October 7, 2022 04:59

Merge branch 'main' into feature/wrr-delete-api

b6ed9e5

imRishN approved these changes Oct 7, 2022

View reviewed changes

Bukhtawar reviewed Oct 12, 2022

View reviewed changes

anshu1106 requested a review from Bukhtawar October 12, 2022 09:12

Merge branch 'main' into feature/wrr-delete-api

6a4c8d5

Bukhtawar approved these changes Oct 13, 2022

View reviewed changes

Bukhtawar merged commit 18f1fa3 into opensearch-project:main Oct 13, 2022

anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Nov 1, 2022

Delete API for weighted round robin search routing (opensearch-projec…

86fff89

…t#4400) * Delete API for weighted round robin search routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 mentioned this pull request Nov 1, 2022

[Backport] Weighted routing 2.x #5008

Merged

6 tasks

anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Nov 3, 2022

Delete API for weighted round robin search routing (opensearch-projec…

cc2080a

…t#4400) * Delete API for weighted round robin search routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 mentioned this pull request Nov 3, 2022

[Backport 2.4]Weighted request routing #5057

Merged

6 tasks

ashking94 pushed a commit to ashking94/OpenSearch that referenced this pull request Nov 7, 2022

Delete API for weighted round robin search routing (opensearch-projec…

342376f

…t#4400) * Delete API for weighted round robin search routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Delete API for weighted round robin search routing #4400

Delete API for weighted round robin search routing #4400

Conversation

anshu1106 commented Sep 3, 2022 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Sep 3, 2022

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gbbafna left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 20, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 22, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 28, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Oct 6, 2022 • edited Loading

Codecov Report

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

anshu1106 commented Oct 6, 2022

github-actions bot commented Oct 7, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 7, 2022

Gradle Check (Jenkins) Run Completed with:

imRishN left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 13, 2022

Gradle Check (Jenkins) Run Completed with:

anshu1106 commented Sep 3, 2022 •

edited

Loading

codecov-commenter commented Oct 6, 2022 •

edited

Loading