Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify diff calculation for remote routing table #15100

Merged
merged 8 commits into from
Sep 2, 2024
Merged

Simplify diff calculation for remote routing table #15100

merged 8 commits into from
Sep 2, 2024

Conversation

Bukhtawar
Copy link
Collaborator

@Bukhtawar Bukhtawar commented Aug 4, 2024 •
edited
Loading

Description

Simplifies diff calculation for remote routing table and makes it compatible with Diff. This change make it simpler to switch between complete diff and incremental diffs.

Some changes in the PR might not be backward compatible with existing Public APIs and would be addressed as a follow up PR

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Bukhtawar
Add Ashish Singh as maintainer
46f2d41 
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
@Bukhtawar Bukhtawar marked this pull request as draft August 4, 2024 01:50
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 3fadc9f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for dd25e86: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 4625fc9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 4cc04c6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for d284c32: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for b1781a0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 607317a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 4607c5d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 9b5c271: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for cfde447: SUCCESS

Merge branch 'main' into main-simplify-rrt
f9567e1 
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for f9567e1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 40c9372: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 11d6e8e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Fix build.
50c98d0 
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 50c98d0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 33535ed: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Merge branch 'main' into main-simplify-rrt.
ea927cc 
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 2, 2024

✅ Gradle check result for ea927cc: SUCCESS

@Bukhtawar Bukhtawar merged commit 7247266 into opensearch-project:main Sep 2, 2024
33 of 34 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 2, 2024
@github-actions
Simplify diff calculation for remote routing table (#15100)
51e135c 
Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Shailendra Singh <singhlhs@amazon.com>
(cherry picked from commit 7247266)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Sep 2, 2024
Merged
himshikha
himshikha reviewed Sep 2, 2024
View reviewed changes
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Comment on lines +538 to +541
&& routingTableDiff.provideDiff() != null
&& (!routingTableDiff.provideDiff().getDiffs().isEmpty()
|| !routingTableDiff.provideDiff().getDeletes().isEmpty()
|| !routingTableDiff.provideDiff().getUpserts().isEmpty())) ? 1 : 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: looks like repeated logic, can we extract this out, or maybe add a method like hasDiff in the routingTableDiff object?

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
updatedIndexRouting.addAll(
remoteRoutingTableService.getUpdatedIndexRoutingTableMetadata(diff.getIndicesRoutingUpdated(), manifest.getIndicesRouting())
);
if (manifest.getCodecVersion() == CODEC_V2 || manifest.getCodecVersion() == CODEC_V3) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just check Codec < V4?

Bukhtawar pushed a commit that referenced this pull request Sep 3, 2024
@opensearch-trigger-bot
Simplify diff calculation for remote routing table (#15100) (#15582)
50f411e 
Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
This was referenced Sep 6, 2024
Open
Open
Open
Open
akolarkunnu pushed a commit to akolarkunnu/OpenSearch that referenced this pull request Sep 10, 2024
@Bukhtawar @akolarkunnu
Simplify diff calculation for remote routing table (opensearch-projec…
9d91758 
…t#15100)

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Shailendra Singh <singhlhs@amazon.com>
This was referenced Sep 11, 2024
Open
Open
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@himshikha himshikha himshikha left review comments

@shailendra0811 shailendra0811 shailendra0811 left review comments

@shwetathareja shwetathareja shwetathareja approved these changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@ashking94 ashking94 Awaiting requested review from ashking94 ashking94 is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@reta reta Awaiting requested review from reta reta is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@jainankitk jainankitk Awaiting requested review from jainankitk jainankitk is a code owner

@linuxpi linuxpi Awaiting requested review from linuxpi linuxpi is a code owner

@gaobinlong gaobinlong Awaiting requested review from gaobinlong

@jed326 jed326 Awaiting requested review from jed326

@peternied peternied Awaiting requested review from peternied

Assignees
No one assigned
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Bukhtawar @himshikha @shwetathareja @shailendra0811