Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Lucene soft-deletes in peer recovery #30522

Merged
merged 46 commits into from
Jun 21, 2018
Merged

Use Lucene soft-deletes in peer recovery #30522

merged 46 commits into from
Jun 21, 2018

Commits on May 10, 2018

  1. Introduce soft-deletes retention policy for peer recovery 

    We are targeting to replace translog by Lucene history in peer-recovery.
In peer-recovery, the primary transfers a safe commit to replicas then
sends subsequent operations after the local checkpoint of that safe
commit. This requires the primary having all operations after the
checkpoint of the current safe commit.

This commit hardens the soft-deletes retention merge policy by taking
safe-commit into account.
    @dnhatn
    dnhatn committed May 10, 2018
    Configuration menu
    Copy the full SHA
    b18eb06 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2018

  1. Don’t check local_checkpoint in commit

    @dnhatn
    dnhatn committed May 12, 2018
    Configuration menu
    Copy the full SHA
    4b2e385 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2018

  1. Prefer synchronized methods

    @dnhatn
    dnhatn committed May 14, 2018
    Configuration menu
    Copy the full SHA
    176b497 View commit details
    Browse the repository at this point in the history

  2. Remove force-merge

    @dnhatn
    dnhatn committed May 14, 2018
    Configuration menu
    Copy the full SHA
    9ae627c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 14, 2018
    Configuration menu
    Copy the full SHA
    ff2215c View commit details
    Browse the repository at this point in the history

Commits on May 15, 2018

  1. Add +1 in 1 place

    @dnhatn
    dnhatn committed May 15, 2018
    Configuration menu
    Copy the full SHA
    188138d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 15, 2018
    Configuration menu
    Copy the full SHA
    6d901bf View commit details
    Browse the repository at this point in the history

  3. acquireTranslogRetentionLock -> acquireRetentionLockForPeerRecovery

    @dnhatn
    dnhatn committed May 15, 2018
    Configuration menu
    Copy the full SHA
    8a78f65 View commit details
    Browse the repository at this point in the history

  4. Cut over the translog snapshot

    @dnhatn
    dnhatn committed May 15, 2018
    Configuration menu
    Copy the full SHA
    6fe8847 View commit details
    Browse the repository at this point in the history

  5. testRecoveryWithOutOfOrderDelete with Lucene history

    @dnhatn
    dnhatn committed May 15, 2018
    Configuration menu
    Copy the full SHA
    cc2b3f0 View commit details
    Browse the repository at this point in the history

  6. Adapt PrimaryReplicaSyncerTests test

    @dnhatn
    dnhatn committed May 15, 2018
    Configuration menu
    Copy the full SHA
    0612a05 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2018

  1. Mute two more tests

    @dnhatn
    dnhatn committed May 16, 2018
    Configuration menu
    Copy the full SHA
    a61d00b View commit details
    Browse the repository at this point in the history

  2. Add discuss

    @dnhatn
    dnhatn committed May 16, 2018
    Configuration menu
    Copy the full SHA
    f3f1fa2 View commit details
    Browse the repository at this point in the history

  3. comment

    @dnhatn
    dnhatn committed May 16, 2018
    Configuration menu
    Copy the full SHA
    65b8458 View commit details
    Browse the repository at this point in the history

  4. Update testRecoveryWithOutOfOrderDelete

    @dnhatn
    dnhatn committed May 16, 2018
    Configuration menu
    Copy the full SHA
    fc3d7d1 View commit details
    Browse the repository at this point in the history

  5. Minor feedback

    @dnhatn
    dnhatn committed May 16, 2018
    Configuration menu
    Copy the full SHA
    bd1b8ac View commit details
    Browse the repository at this point in the history

Commits on May 17, 2018

  1. Fix recovery tests

    @dnhatn
    dnhatn committed May 17, 2018
    Configuration menu
    Copy the full SHA
    b1e73aa View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 17, 2018
    Configuration menu
    Copy the full SHA
    f7ea71c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 17, 2018
    Configuration menu
    Copy the full SHA
    04112c6 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 23, 2018
    Configuration menu
    Copy the full SHA
    e34154a View commit details
    Browse the repository at this point in the history

Commits on May 24, 2018

  1. use retention policy to decide seq# or file-based

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    1531024 View commit details
    Browse the repository at this point in the history

  2. Fix an engine test

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    86c3eba View commit details
    Browse the repository at this point in the history

  3. fix test - wait for flush

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    b3d0d5f View commit details
    Browse the repository at this point in the history

  4. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    8320647 View commit details
    Browse the repository at this point in the history

  5. bootstrap from max_seqno from an empty commit

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    6b95e21 View commit details
    Browse the repository at this point in the history

  6. harden bootstrap

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    3be0e30 View commit details
    Browse the repository at this point in the history

  7. naming

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    ca3f781 View commit details
    Browse the repository at this point in the history

  8. getMaxExposedSeqNoToMergePolicy -> getMaxExposedSeqNo

    @dnhatn
    dnhatn committed May 24, 2018
    Configuration menu
    Copy the full SHA
    65ede0b View commit details
    Browse the repository at this point in the history

Commits on May 25, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 25, 2018
    Configuration menu
    Copy the full SHA
    a0e58b9 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2018

  1. decouple min_retained seqno from merge policy

    @dnhatn
    dnhatn committed May 28, 2018
    Configuration menu
    Copy the full SHA
    c1e03d1 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 28, 2018
    Configuration menu
    Copy the full SHA
    c5ba76f View commit details
    Browse the repository at this point in the history

  3. simplify tests

    @dnhatn
    dnhatn committed May 28, 2018
    Configuration menu
    Copy the full SHA
    33be718 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 31, 2018
    Configuration menu
    Copy the full SHA
    c717bf7 View commit details
    Browse the repository at this point in the history

  2. minSeqNo -> startingSeqNo

    @dnhatn
    dnhatn committed May 31, 2018
    Configuration menu
    Copy the full SHA
    df132e1 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 31, 2018
    Configuration menu
    Copy the full SHA
    1bcd443 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed May 31, 2018
    Configuration menu
    Copy the full SHA
    f232c8a View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    591d521 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    76a035f View commit details
    Browse the repository at this point in the history

  2. Adjust testForceMergeWithSoftDeletesRetentionAndRecoverySource

    @dnhatn
    dnhatn committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    78c0d92 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 11, 2018
    Configuration menu
    Copy the full SHA
    c30de4a View commit details
    Browse the repository at this point in the history

  2. overriddenOperations -> skippedOperations

    @dnhatn
    dnhatn committed Jun 11, 2018
    Configuration menu
    Copy the full SHA
    5ff18f9 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 15, 2018
    Configuration menu
    Copy the full SHA
    8a37126 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 15, 2018
    Configuration menu
    Copy the full SHA
    88950b3 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 19, 2018
    Configuration menu
    Copy the full SHA
    dbe0472 View commit details
    Browse the repository at this point in the history

  2. Naming

    @dnhatn
    dnhatn committed Jun 19, 2018
    Configuration menu
    Copy the full SHA
    f9eeb90 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2018

  1. Merge branch 'ccr' into safecommit-mergepolicy

    @dnhatn
    dnhatn committed Jun 21, 2018
    Configuration menu
    Copy the full SHA
    ccb6e80 View commit details
    Browse the repository at this point in the history