Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement parallel recovery #97

Merged
merged 16 commits into from
Sep 16, 2021
Merged

Implement parallel recovery #97

merged 16 commits into from
Sep 16, 2021

Commits on Sep 8, 2021

  1. Implement parallel recovery 

    + Add config `recovery_threads` to control concurrency of recovery.
+ Implement parallel recovery & apply.
+ Update unit tests.

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 8, 2021
    Configuration menu
    Copy the full SHA
    1fdc968 View commit details
    Browse the repository at this point in the history

  2. Refine parallel impl 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 8, 2021
    Configuration menu
    Copy the full SHA
    3e97c75 View commit details
    Browse the repository at this point in the history

  3. Remove unused comments 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 8, 2021
    Configuration menu
    Copy the full SHA
    506ddbc View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2021

  1. Introduce PipeLog::SequentialReplayMachine trait 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 9, 2021
    Configuration menu
    Copy the full SHA
    028ea60 View commit details
    Browse the repository at this point in the history

  2. Minor cleanup. 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 9, 2021
    Configuration menu
    Copy the full SHA
    fd2349f View commit details
    Browse the repository at this point in the history

  3. Truncate activate file at valid offset 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 9, 2021
    Configuration menu
    Copy the full SHA
    fa6d579 View commit details
    Browse the repository at this point in the history

  4. Fix recovery when given threads is 1 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 9, 2021
    Configuration menu
    Copy the full SHA
    64507db View commit details
    Browse the repository at this point in the history

  5. cleanup 

    Signed-off-by: tabokie <xy.tao@outlook.com>
    @tabokie
    tabokie committed Sep 9, 2021
    Configuration menu
    Copy the full SHA
    4dbfe8f View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2021

  1. cleanup file_pipe_log.rs 

    Signed-off-by: tabokie <xy.tao@outlook.com>
    @tabokie
    tabokie committed Sep 10, 2021
    Configuration menu
    Copy the full SHA
    b5909ee View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/tikv/raft-engine into tmp 

    Signed-off-by: tabokie <xy.tao@outlook.com>
    @tabokie
    tabokie committed Sep 10, 2021
    Configuration menu
    Copy the full SHA
    a229818 View commit details
    Browse the repository at this point in the history

  3. keep rayon version consistent with TiKV 

    Signed-off-by: tabokie <xy.tao@outlook.com>
    @tabokie
    tabokie committed Sep 10, 2021
    Configuration menu
    Copy the full SHA
    69c5fed View commit details
    Browse the repository at this point in the history

  4. simplify FileBuilder trait returning type 

    Signed-off-by: tabokie <xy.tao@outlook.com>
    @tabokie
    tabokie committed Sep 10, 2021
    Configuration menu
    Copy the full SHA
    cdf26fa View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2021

  1. adapt bench to new engien interfaces 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 14, 2021
    Configuration menu
    Copy the full SHA
    5044256 View commit details
    Browse the repository at this point in the history

  2. add 10GB bench to default config 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 14, 2021
    Configuration menu
    Copy the full SHA
    67159de View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2021

  1. add unit test for parallel recover merge operation 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    63ac7a9 View commit details
    Browse the repository at this point in the history

  2. add unit test for merged stats 

    Signed-off-by: MrCroxx <mrcroxx@outlook.com>
    @MrCroxx
    MrCroxx committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    4c85b99 View commit details
    Browse the repository at this point in the history