From 729388e02ed2518a6f4ac62866a2e433dcf627c4 Mon Sep 17 00:00:00 2001 From: Haze188 Date: Wed, 19 Jun 2024 16:19:36 +0800 Subject: [PATCH] [MoE] Resolve .github conflict (#5829) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [Fix/Example] Fix Llama Inference Loading Data Type (#5763) * [fix/example] fix llama inference loading dtype * revise loading dtype of benchmark llama3 * [release] update version (#5752) * [release] update version * [devops] update compatibility test * [devops] update compatibility test * [devops] update compatibility test * [devops] update compatibility test * [test] fix ddp plugin test * [test] fix gptj and rpc test * [devops] fix cuda ext compatibility * [inference] fix flash decoding test * [inference] fix flash decoding test * fix (#5765) * [test] Fix/fix testcase (#5770) * [fix] branch for fix testcase; * [fix] fix test_analyzer & test_auto_parallel; * [fix] remove local change about moe; * [fix] rm local change moe; * [Hotfix] Add missing init file in inference.executor (#5774) * [CI/tests] simplify some test case to reduce testing time (#5755) * [ci/tests] simplify some test case to reduce testing time * [ci/tests] continue to remove test case to reduce ci time cost * restore some test config * [ci/tests] continue to reduce ci time cost * [misc] update dockerfile (#5776) * [misc] update dockerfile * [misc] update dockerfile * [devops] fix docker ci (#5780) * [Inference]Add Streaming LLM (#5745) * Add Streaming LLM * add some parameters to llama_generation.py * verify streamingllm config * add test_streamingllm.py * modified according to the opinions of review * add Citation * change _block_tables tolist * [hotfix] fix llama flash attention forward (#5777) * [misc] Accelerate CI for zero and dist optim (#5758) * remove fp16 from lamb * remove d2h copy in checking states --------- Co-authored-by: Edenzzzz * [Test/CI] remove test cases to reduce CI duration (#5753) * [test] smaller gpt2 test case * [test] reduce test cases: tests/test_zero/test_gemini/test_zeroddp_state_dict.py * [test] reduce test cases: tests/test_zero/test_gemini/test_grad_accum.py * [test] reduce test cases tests/test_zero/test_gemini/test_optim.py * Revert "[test] smaller gpt2 test case" Some tests might depend on the size of model (num of chunks) This reverts commit df705a5210b8901645992adf276e320e48766ebf. * [test] reduce test cases: tests/test_checkpoint_io/test_gemini_checkpoint_io.py * [CI] smaller test model for two mwo the two modifid cases * [CI] hardcode gpt model for tests/test_zero/test_gemini/test_search.py since we need a fixed answer there * [hotfix] fix testcase in test_fx/test_tracer (#5779) * [fix] branch for fix testcase; * [fix] fix test_analyzer & test_auto_parallel; * [fix] remove local change about moe; * [fix] rm local change moe; * [fix] fix test_deepfm_model & test_dlrf_model； * [fix] fix test_hf_albert & test_hf_gpt; * [gemini] optimize reduce scatter d2h copy (#5760) * [gemini] optimize reduce scatter d2h copy * [fix] fix missing reduce variable * [refactor] remove legacy async reduce scatter code * [gemini] missing sync * Revert "[refactor] remove legacy async reduce scatter code" This reverts commit 58ad76d4665032bbe548d066116d1c572ce98979. * [gemini] further optimize with async all reduce * [fix] pass flag from manager to chunk * Allow building cuda extension without a device. (#5535) Added FORCE_CUDA environment variable support, to enable building extensions where a GPU device is not present but cuda libraries are. * [misc] fix dist logger (#5782) * [install]fix setup (#5786) * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [misc] update requirements (#5787) * [shardformer] fix import (#5788) * upgrade colossal-chat support tp_group>1, add sp for sft * upgrade ppo dpo rm script * run pre-commit * moupdate ci tests, st ci test cases passed, tp failed in generation for ppo, sp is buggy * fix training script * fix ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix transformers version * remove duplicated test * fix datasets version * remove models that require huggingface auth from ci * remove local data path * update ci * remove baichuan from template test due to transformer version conflict * merge * Refactor modeling by adding attention backend Signed-off-by: char-1ee * Fix tests and naming Signed-off-by: char-1ee * Pass inference model shard configs for module init Signed-off-by: char-1ee * Clean up Signed-off-by: char-1ee * replace the customized dataloader setup with the build-in one * replace the customized dataloader setup with the build-in one * Remove flash attention backend Signed-off-by: char-1ee * fix readme * Fix test import Signed-off-by: char-1ee * update sft trainning script * [Inference]refactor baichuan (#5791) * refactor baichuan * remove unused code and add TODO for lazyinit * [test] fix chatglm test kit (#5793) * [shardformer] fix modeling of bloom and falcon (#5796) * [test] fix qwen2 pytest distLarge (#5797) * [Inference] Fix flash-attn import and add model test (#5794) * Fix torch int32 dtype Signed-off-by: char-1ee * Fix flash-attn import Signed-off-by: char-1ee * Add generalized model test Signed-off-by: char-1ee * Remove exposed path to model Signed-off-by: char-1ee * Add default value for use_flash_attn Signed-off-by: char-1ee * Rename model test Signed-off-by: char-1ee --------- Signed-off-by: char-1ee * [Gemini] Use async stream to prefetch and h2d data moving (#5781) * use async stream to prefetch and h2d data moving * Remove redundant code * [gemini] quick fix on possible async operation (#5803) * [gemini] quick fix on possible async operation * [gemini] quick fix on possible async operation * [shardformer] upgrade transformers to 4.39.3 (#5815) * [shardformer]upgrade transformers for gpt2/gptj/whisper (#5807) * [shardformer] fix modeling of gpt2 and gptj * [shardformer] fix whisper modeling * [misc] update requirements --------- Co-authored-by: ver217 * [shardformer]upgrade transformers for mistral (#5808) * upgrade transformers for mistral * fix * fix * [shardformer]upgrade transformers for llama (#5809) * update transformers fix * fix * fix * [inference] upgrade transformers (#5810) * update transformers fix * fix * fix * fix * fix * [gemini] update transformers for gemini (#5814) --------- Co-authored-by: ver217 * Support 4d parallel + flash attention (#5789) * support tp + sp + pp * remove comments --------- Co-authored-by: Edenzzzz --------- Signed-off-by: char-1ee Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com> Co-authored-by: Hongxin Liu Co-authored-by: flybird11111 <1829166702@qq.com> Co-authored-by: duanjunwen <935724073@qq.com> Co-authored-by: yuehuayingxueluo <867460659@qq.com> Co-authored-by: Edenzzzz Co-authored-by: Edenzzzz Co-authored-by: botbw Co-authored-by: Charles Coulombe Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: YeAnbang Co-authored-by: char-1ee Co-authored-by: Runyu Lu <77330637+LRY89757@users.noreply.github.com> Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com> Co-authored-by: Guangyao Zhang --- .github/workflows/compatiblity_test_on_dispatch.yml | 4 ++-- .github/workflows/compatiblity_test_on_pr.yml | 4 ++-- .github/workflows/compatiblity_test_on_schedule.yml | 4 ++-- .github/workflows/release_docker_after_publish.yml | 2 ++ .github/workflows/run_chatgpt_examples.yml | 11 ++++++----- .github/workflows/run_chatgpt_unit_tests.yml | 11 +++++------ 6 files changed, 19 insertions(+), 17 deletions(-) diff --git a/.github/workflows/compatiblity_test_on_dispatch.yml b/.github/workflows/compatiblity_test_on_dispatch.yml index 3da8b5e77df9..3eee564c29ea 100644 --- a/.github/workflows/compatiblity_test_on_dispatch.yml +++ b/.github/workflows/compatiblity_test_on_dispatch.yml @@ -51,11 +51,11 @@ jobs: container: image: ${{ matrix.container }} options: --gpus all --rm -v /dev/shm -v /data/scratch/:/data/scratch/ - timeout-minutes: 120 + timeout-minutes: 200 steps: - name: Install dependencies run: | - pip install -U pip setuptools wheel --user + pip install -U pip setuptools==68.2.2 wheel --user - uses: actions/checkout@v2 with: repository: hpcaitech/TensorNVMe diff --git a/.github/workflows/compatiblity_test_on_pr.yml b/.github/workflows/compatiblity_test_on_pr.yml index 10ac0e128dc6..b418c843e7f6 100644 --- a/.github/workflows/compatiblity_test_on_pr.yml +++ b/.github/workflows/compatiblity_test_on_pr.yml @@ -42,14 +42,14 @@ jobs: container: image: ${{ matrix.container }} options: --gpus all --rm -v /dev/shm -v /data/scratch/:/data/scratch/ - timeout-minutes: 120 + timeout-minutes: 200 concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}-run-test-${{ matrix.container }} cancel-in-progress: true steps: - name: Install dependencies run: | - pip install -U pip setuptools wheel --user + pip install -U pip setuptools==68.2.2 wheel --user - uses: actions/checkout@v2 with: repository: hpcaitech/TensorNVMe diff --git a/.github/workflows/compatiblity_test_on_schedule.yml b/.github/workflows/compatiblity_test_on_schedule.yml index 84ea7e28d967..8d98e775c828 100644 --- a/.github/workflows/compatiblity_test_on_schedule.yml +++ b/.github/workflows/compatiblity_test_on_schedule.yml @@ -39,11 +39,11 @@ jobs: container: image: ${{ matrix.container }} options: --gpus all --rm -v /dev/shm -v /data/scratch/:/data/scratch/ - timeout-minutes: 120 + timeout-minutes: 200 steps: - name: Install dependencies run: | - pip install -U pip setuptools wheel --user + pip install -U pip setuptools==68.2.2 wheel --user - uses: actions/checkout@v2 with: diff --git a/.github/workflows/release_docker_after_publish.yml b/.github/workflows/release_docker_after_publish.yml index 0792544bf403..23aac9b544b0 100644 --- a/.github/workflows/release_docker_after_publish.yml +++ b/.github/workflows/release_docker_after_publish.yml @@ -28,6 +28,8 @@ jobs: docker tag $tag $latest echo "tag=${tag}" >> $GITHUB_OUTPUT echo "latest=${latest}" >> $GITHUB_OUTPUT + env: + DOCKER_BUILDKIT: 0 - name: Log in to Docker Hub uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9 diff --git a/.github/workflows/run_chatgpt_examples.yml b/.github/workflows/run_chatgpt_examples.yml index ba997f144cd7..4ea86b609267 100644 --- a/.github/workflows/run_chatgpt_examples.yml +++ b/.github/workflows/run_chatgpt_examples.yml @@ -4,10 +4,11 @@ on: pull_request: types: [synchronize, opened, reopened] paths: - - "applications/Chat/coati/**" - - "applications/Chat/requirements.txt" - - "applications/Chat/setup.py" - - "applications/Chat/examples/**" + - "applications/ColossalChat/coati/**" + - "applications/ColossalChat/requirements.txt" + - "applications/ColossalChat/setup.py" + - "applications/ColossalChat/examples/**" + - "applications/ColossalChat/tests/**" jobs: tests: @@ -41,7 +42,7 @@ jobs: - name: Install Transformers run: | - pip install transformers==4.34.1 + pip install transformers==4.36.2 - name: Execute Examples run: | diff --git a/.github/workflows/run_chatgpt_unit_tests.yml b/.github/workflows/run_chatgpt_unit_tests.yml index 1d8a53e4feed..c0e74ecbbab0 100644 --- a/.github/workflows/run_chatgpt_unit_tests.yml +++ b/.github/workflows/run_chatgpt_unit_tests.yml @@ -4,12 +4,11 @@ on: pull_request: types: [synchronize, opened, reopened] paths: - - 'applications/Chat/coati/**' - - 'applications/Chat/requirements.txt' - - 'applications/Chat/setup.py' - - 'applications/Chat/requirements-test.txt' - - 'applications/Chat/tests/**' - - 'applications/Chat/pytest.ini' + - 'applications/ColossalChat/coati/**' + - 'applications/ColossalChat/requirements.txt' + - 'applications/ColossalChat/setup.py' + - 'applications/ColossalChat/tests/**' + - 'applications/ColossalChat/pytest.ini' jobs: tests: