Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[shardformer] Support 4D parallel + Flash Attention #5789

Merged
merged 12 commits into from
Jun 17, 2024
Merged

[shardformer] Support 4D parallel + Flash Attention #5789

merged 12 commits into from
Jun 17, 2024

Commits on Jun 6, 2024

  1. support tp + sp + pp

    Edenzzzz committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    95d0a27 View commit details
    Browse the repository at this point in the history

  2. remove comments

    Edenzzzz committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    d2aeb2e View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into sp_flash_attn

    @Edenzzzz
    Edenzzzz authored Jun 6, 2024
    Configuration menu
    Copy the full SHA
    8ffde6b View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    932753f View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. fix sp group reversed bug

    Edenzzzz committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    c92951d View commit details
    Browse the repository at this point in the history

  2. remove model forward replacement with pp

    Edenzzzz committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    6d6eb37 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. debug

    Edenzzzz committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    90485ce View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2024

  1. merge with main

    Edenzzzz committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    c088d6d View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2024

  1. fix attention mask bug

    Edenzzzz committed Jun 16, 2024
    Configuration menu
    Copy the full SHA
    996944d View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. fix sp_size bug

    Edenzzzz committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    20059ae View commit details
    Browse the repository at this point in the history

  2. remove comments

    Edenzzzz committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    a2e07c9 View commit details
    Browse the repository at this point in the history

  3. add tab

    Edenzzzz committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    07849f6 View commit details
    Browse the repository at this point in the history