Running Example on Free T4 GPU through Google Colab

[hammad93]

Please check that this issue hasn't been reported before.

• I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

The setup or first 2 cells in the notebook can install the environment. This set of installation code should run all subsequent code to successfully recreate the training and inference steps of LLM's through Axolotl. https://github.com/axolotl-ai-cloud/axolotl/blob/main/examples/colab-notebooks/colab-axolotl-example.ipynb

Current behaviour

The first cell is successful,

import torch
# Check so there is a gpu available, a T4(free tier) is enough to run this notebook
assert (torch.cuda.is_available()==True)

The second cell never completes and even if we try to change some of the code, the accelerate command doesn't work. It hangs at flash attention.

!pip install -e git+https://github.com/axolotl-ai-cloud/axolotl#egg=axolotl
!pip install flash-attn=="2.5.0"
!pip install deepspeed=="0.13.1"!pip install mlflow=="2.13.0"

Here's the tail of the installation output from the cell,

:
:
  Attempting uninstall: gcsfs
    Found existing installation: gcsfs 2024.6.1
    Uninstalling gcsfs-2024.6.1:
      Successfully uninstalled gcsfs-2024.6.1
  Running setup.py develop for axolotl
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
albumentations 1.4.14 requires pydantic>=2.7.0, but you have pydantic 2.6.3 which is incompatible.
cudf-cu12 24.4.1 requires pyarrow<15.0.0a0,>=14.0.1, but you have pyarrow 17.0.0 which is incompatible.
ibis-framework 8.0.0 requires pyarrow<16,>=2, but you have pyarrow 17.0.0 which is incompatible.
Successfully installed accelerate-0.34.2 addict-2.4.0 aiobotocore-2.14.0 aiofiles-23.2.1 aioitertools-0.12.0 art-6.2 autoawq-0.2.5 autoawq-kernels-0.0.6 axolotl-0.4.1 bitsandbytes-0.43.3 botocore-1.35.7 colorama-0.4.6 coloredlogs-15.0.1 datasets-2.20.0 dill-0.3.8 docker-pycreds-0.4.0 evaluate-0.4.1 fastapi-0.114.0 ffmpy-0.4.0 fire-0.6.0 fschat-0.2.36 fsspec-2024.5.0 gcsfs-2024.5.0 gitdb-4.0.11 gitpython-3.1.43 gradio-3.50.2 gradio-client-0.6.1 h11-0.14.0 hf_transfer-0.1.8 httpcore-1.0.5 httpx-0.27.2 humanfriendly-10.0 jmespath-1.0.1 latex2mathml-3.77.0 liger-kernel-0.2.1 markdown2-2.5.0 multiprocess-0.70.16 nh3-0.2.18 optimum-1.16.2 orjson-3.10.7 packaging-23.2 peft-0.12.0 pyarrow-17.0.0 pydantic-2.6.3 pydantic-core-2.16.3 pydub-0.25.1 pynvml-11.5.3 python-dotenv-1.0.1 python-multipart-0.0.9 responses-0.18.0 s3fs-2024.5.0 scikit-learn-1.4.2 semantic-version-2.10.0 sentry-sdk-2.13.0 setproctitle-1.3.3 shortuuid-1.0.13 shtab-1.7.1 smmap-5.0.1 starlette-0.38.5 svgwrite-1.4.3 tiktoken-0.7.0 triton-3.0.0 trl-0.9.6 tyro-0.8.10 uvicorn-0.30.6 wandb-0.17.9 wavedrom-2.0.3.post3 websockets-11.0.3 xformers-0.0.27.post2 xxhash-3.5.0 zstandard-0.22.0
Collecting flash-attn==2.5.0
  Downloading flash_attn-2.5.0.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 38.5 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (2.4.0+cu121)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (0.8.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (23.2)
Collecting ninja (from flash-attn==2.5.0)
  Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl.metadata (5.3 kB)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.15.4)
Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (4.12.2)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (1.13.2)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.1.4)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (2024.5.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->flash-attn==2.5.0) (2.1.5)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->flash-attn==2.5.0) (1.3.0)
Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 23.2 MB/s eta 0:00:00
Building wheels for collected packages: flash-attn

Steps to reproduce

1. Open https://colab.research.google.com/ and login if needed
2. Click open Github, type in axolotl-ai-cloud for organization and select axolotl for repository
3. Select the example notebook
4. Run the cells in the notebook for installation

Config yaml

base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer

load_in_8bit: false
load_in_4bit: true
strict: false

datasets:
  - path: mhenrichsen/alpaca_2k_test
    type: alpaca
dataset_prepared_path:
val_set_size: 0.05
output_dir: ./outputs/qlora-out

adapter: qlora
lora_model_dir:

sequence_len: 4096
sample_packing: true
eval_sample_packing: false
pad_to_sequence_len: true

lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:

wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 4
optimizer: paged_adamw_32bit
lr_scheduler: cosine
learning_rate: 0.0002

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 10
evals_per_epoch: 4
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:

Possible solution

This might be because the T4 GPU is not supported by this library. Here's the documentation about which GPU's are supported, https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#nvidia-cuda-support

I used the instructions in the README instead to install and train TinyLLAMA using Axolotl instead. I also changed the config to set the flash attention layer to false. This still installs the flash-attn somehow but we get RuntimeError: FlashAttention only supports Ampere GPUs or newer error at runtime instead.

This cell will install it,

!git clone https://github.com/axolotl-ai-cloud/axolotl
%cd axolotl
!pip3 install packaging ninja
!pip3 install -e '.[flash-attn,deepspeed]'

Then in the config.yaml,

flash_attention: false

This does run on the free T4 GPU but still takes hours to finish and may need a different config.

Which Operating Systems are you using?

• Linux
• macOS
• Windows

Python Version

3.10

axolotl branch-commit

main

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this bug has not been reported yet.
• I am using the latest version of axolotl.
• I have provided enough information for the maintainers to reproduce and diagnose the issue.

[gj414c]

I can confirm this. Here is the full report with the error message as well:

Obtaining axolotl from git+https://github.com/axolotl-ai-cloud/axolotl#egg=axolotl
  Cloning https://github.com/axolotl-ai-cloud/axolotl to ./src/axolotl
  Running command git clone --filter=blob:none --quiet https://github.com/axolotl-ai-cloud/axolotl /content/src/axolotl
  Resolved https://github.com/axolotl-ai-cloud/axolotl to commit 3853ab7ae9220dfbd78cd628e54fde75fb89df97
  Preparing metadata (setup.py) ... done
Collecting fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe (from axolotl)
  Cloning https://github.com/lm-sys/FastChat.git (to revision 27a05b04a35510afb1d767ae7e5990cbd278f8fe) to /tmp/pip-install-e8ty8535/fschat_45e1dde96ea84f9b8bcecc6a4cace483
  Running command git clone --filter=blob:none --quiet https://github.com/lm-sys/FastChat.git /tmp/pip-install-e8ty8535/fschat_45e1dde96ea84f9b8bcecc6a4cace483
  Running command git rev-parse -q --verify 'sha^27a05b04a35510afb1d767ae7e5990cbd278f8fe'
  Running command git fetch -q https://github.com/lm-sys/FastChat.git 27a05b04a35510afb1d767ae7e5990cbd278f8fe
  Running command git checkout -q 27a05b04a35510afb1d767ae7e5990cbd278f8fe
  Resolved https://github.com/lm-sys/FastChat.git to commit 27a05b04a35510afb1d767ae7e5990cbd278f8fe
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting packaging==23.2 (from axolotl)
  Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting peft==0.12.0 (from axolotl)
  Downloading peft-0.12.0-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: transformers==4.44.2 in /usr/local/lib/python3.10/dist-packages (from axolotl) (4.44.2)
Requirement already satisfied: tokenizers>=0.19.1 in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.19.1)
Collecting bitsandbytes==0.43.3 (from axolotl)
  Downloading bitsandbytes-0.43.3-py3-none-manylinux_2_24_x86_64.whl.metadata (3.5 kB)
Requirement already satisfied: accelerate==0.34.2 in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.34.2)
Collecting datasets==2.20.0 (from axolotl)
  Downloading datasets-2.20.0-py3-none-any.whl.metadata (19 kB)
Collecting pydantic==2.6.3 (from axolotl)
  Downloading pydantic-2.6.3-py3-none-any.whl.metadata (84 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.4/84.4 kB 7.2 MB/s eta 0:00:00
Collecting addict (from axolotl)
  Downloading addict-2.4.0-py3-none-any.whl.metadata (1.0 kB)
Collecting fire (from axolotl)
  Downloading fire-0.6.0.tar.gz (88 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.4/88.4 kB 8.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: PyYAML>=6.0 in /usr/local/lib/python3.10/dist-packages (from axolotl) (6.0.2)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from axolotl) (2.32.3)
Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.1.99)
Collecting wandb (from axolotl)
  Downloading wandb-0.18.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.6 kB)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.8.0)
Collecting optimum==1.16.2 (from axolotl)
  Downloading optimum-1.16.2-py3-none-any.whl.metadata (17 kB)
Collecting hf_transfer (from axolotl)
  Downloading hf_transfer-0.1.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB)
Collecting colorama (from axolotl)
  Downloading colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)
Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.60.0)
Requirement already satisfied: numpy<=2.0.1,>=1.24.4 in /usr/local/lib/python3.10/dist-packages (from axolotl) (1.26.4)
Collecting evaluate==0.4.1 (from axolotl)
  Downloading evaluate-0.4.1-py3-none-any.whl.metadata (9.4 kB)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from axolotl) (1.13.1)
Collecting scikit-learn==1.4.2 (from axolotl)
  Downloading scikit_learn-1.4.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Collecting pynvml (from axolotl)
  Downloading pynvml-11.5.3-py3-none-any.whl.metadata (8.8 kB)
Collecting art (from axolotl)
  Downloading art-6.2-py3-none-any.whl.metadata (69 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 69.2/69.2 kB 6.2 MB/s eta 0:00:00
Collecting gradio==3.50.2 (from axolotl)
  Downloading gradio-3.50.2-py3-none-any.whl.metadata (17 kB)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from axolotl) (2.17.0)
Collecting python-dotenv==1.0.1 (from axolotl)
  Downloading python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)
Collecting autoawq>=0.2.5 (from axolotl)
  Downloading autoawq-0.2.6-cp310-cp310-manylinux2014_x86_64.whl.metadata (18 kB)
Collecting triton>=2.3.0 (from axolotl)
  Downloading triton-3.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.3 kB)
Collecting liger-kernel==0.2.1 (from axolotl)
  Downloading liger_kernel-0.2.1-py3-none-any.whl.metadata (17 kB)
Collecting s3fs>=2024.5.0 (from axolotl)
  Downloading s3fs-2024.9.0-py3-none-any.whl.metadata (1.6 kB)
Requirement already satisfied: gcsfs>=2024.5.0 in /usr/local/lib/python3.10/dist-packages (from axolotl) (2024.6.1)
Collecting trl==0.9.6 (from axolotl)
  Downloading trl-0.9.6-py3-none-any.whl.metadata (12 kB)
Collecting zstandard==0.22.0 (from axolotl)
  Downloading zstandard-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.9 kB)
Requirement already satisfied: fastcore in /usr/local/lib/python3.10/dist-packages (from axolotl) (1.7.5)
Requirement already satisfied: torch==2.4.0+cu121 in /usr/local/lib/python3.10/dist-packages (from axolotl) (2.4.0+cu121)
Collecting xformers>=0.0.26.post1 (from axolotl)
  Downloading xformers-0.0.28-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (1.0 kB)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.34.2->axolotl) (5.9.5)
Requirement already satisfied: huggingface-hub>=0.21.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.34.2->axolotl) (0.24.6)
Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.34.2->axolotl) (0.4.5)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets==2.20.0->axolotl) (3.16.0)
Collecting pyarrow>=15.0.0 (from datasets==2.20.0->axolotl)
  Downloading pyarrow-17.0.0-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (3.3 kB)
Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets==2.20.0->axolotl) (0.6)
Collecting dill<0.3.9,>=0.3.0 (from datasets==2.20.0->axolotl)
  Downloading dill-0.3.8-py3-none-any.whl.metadata (10 kB)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets==2.20.0->axolotl) (2.1.4)
Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.10/dist-packages (from datasets==2.20.0->axolotl) (4.66.5)
Collecting xxhash (from datasets==2.20.0->axolotl)
  Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting multiprocess (from datasets==2.20.0->axolotl)
  Downloading multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)
Collecting fsspec<=2024.5.0,>=2023.1.0 (from fsspec[http]<=2024.5.0,>=2023.1.0->datasets==2.20.0->axolotl)
  Downloading fsspec-2024.5.0-py3-none-any.whl.metadata (11 kB)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets==2.20.0->axolotl) (3.10.5)
Collecting responses<0.19 (from evaluate==0.4.1->axolotl)
  Downloading responses-0.18.0-py3-none-any.whl.metadata (29 kB)
Collecting aiofiles<24.0,>=22.0 (from gradio==3.50.2->axolotl)
  Downloading aiofiles-23.2.1-py3-none-any.whl.metadata (9.7 kB)
Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (4.2.2)
Collecting fastapi (from gradio==3.50.2->axolotl)
  Downloading fastapi-0.114.1-py3-none-any.whl.metadata (27 kB)
Collecting ffmpy (from gradio==3.50.2->axolotl)
  Downloading ffmpy-0.4.0-py3-none-any.whl.metadata (2.9 kB)
Collecting gradio-client==0.6.1 (from gradio==3.50.2->axolotl)
  Downloading gradio_client-0.6.1-py3-none-any.whl.metadata (7.1 kB)
Collecting httpx (from gradio==3.50.2->axolotl)
  Downloading httpx-0.27.2-py3-none-any.whl.metadata (7.1 kB)
Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (6.4.5)
Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (3.1.4)
Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (2.1.5)
Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (3.7.1)
Collecting orjson~=3.0 (from gradio==3.50.2->axolotl)
  Downloading orjson-3.10.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.4/50.4 kB 3.5 MB/s eta 0:00:00
Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (9.4.0)
Collecting pydub (from gradio==3.50.2->axolotl)
  Downloading pydub-0.25.1-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting python-multipart (from gradio==3.50.2->axolotl)
  Downloading python_multipart-0.0.9-py3-none-any.whl.metadata (2.5 kB)
Collecting semantic-version~=2.0 (from gradio==3.50.2->axolotl)
  Downloading semantic_version-2.10.0-py2.py3-none-any.whl.metadata (9.7 kB)
Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (4.12.2)
Collecting uvicorn>=0.14.0 (from gradio==3.50.2->axolotl)
  Downloading uvicorn-0.30.6-py3-none-any.whl.metadata (6.6 kB)
Collecting websockets<12.0,>=10.0 (from gradio==3.50.2->axolotl)
  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting coloredlogs (from optimum==1.16.2->axolotl)
  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from optimum==1.16.2->axolotl) (1.13.2)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic==2.6.3->axolotl) (0.7.0)
Collecting pydantic-core==2.16.3 (from pydantic==2.6.3->axolotl)
  Downloading pydantic_core-2.16.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.5 kB)
Requirement already satisfied: joblib>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn==1.4.2->axolotl) (1.4.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn==1.4.2->axolotl) (3.5.0)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.4.0+cu121->axolotl) (3.3)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.44.2->axolotl) (2024.5.15)
Collecting tyro>=0.5.11 (from trl==0.9.6->axolotl)
  Downloading tyro-0.8.10-py3-none-any.whl.metadata (8.4 kB)
INFO: pip is looking at multiple versions of autoawq to determine which version is compatible with other requirements. This could take a while.
Collecting autoawq>=0.2.5 (from axolotl)
  Downloading autoawq-0.2.5-cp310-cp310-manylinux2014_x86_64.whl.metadata (16 kB)
Collecting autoawq-kernels (from autoawq>=0.2.5->axolotl)
  Downloading autoawq_kernels-0.0.8-cp310-cp310-manylinux2014_x86_64.whl.metadata (2.4 kB)
Requirement already satisfied: decorator>4.1.2 in /usr/local/lib/python3.10/dist-packages (from gcsfs>=2024.5.0->axolotl) (4.4.2)
INFO: pip is looking at multiple versions of gcsfs to determine which version is compatible with other requirements. This could take a while.
Collecting gcsfs>=2024.5.0 (from axolotl)
  Downloading gcsfs-2024.9.0.post1-py2.py3-none-any.whl.metadata (1.6 kB)
  Downloading gcsfs-2024.6.0-py2.py3-none-any.whl.metadata (1.6 kB)
  Downloading gcsfs-2024.5.0-py2.py3-none-any.whl.metadata (1.6 kB)
Requirement already satisfied: google-auth>=1.2 in /usr/local/lib/python3.10/dist-packages (from gcsfs>=2024.5.0->axolotl) (2.27.0)
Requirement already satisfied: google-auth-oauthlib in /usr/local/lib/python3.10/dist-packages (from gcsfs>=2024.5.0->axolotl) (1.2.1)
Requirement already satisfied: google-cloud-storage in /usr/local/lib/python3.10/dist-packages (from gcsfs>=2024.5.0->axolotl) (2.8.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->axolotl) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->axolotl) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->axolotl) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->axolotl) (2024.8.30)
Collecting aiobotocore<3.0.0,>=2.5.4 (from s3fs>=2024.5.0->axolotl)
  Downloading aiobotocore-2.15.0-py3-none-any.whl.metadata (23 kB)
INFO: pip is looking at multiple versions of s3fs to determine which version is compatible with other requirements. This could take a while.
Collecting s3fs>=2024.5.0 (from axolotl)
  Downloading s3fs-2024.6.1-py3-none-any.whl.metadata (1.6 kB)
  Downloading s3fs-2024.6.0-py3-none-any.whl.metadata (1.6 kB)
  Downloading s3fs-2024.5.0-py3-none-any.whl.metadata (1.6 kB)
INFO: pip is looking at multiple versions of xformers to determine which version is compatible with other requirements. This could take a while.
Collecting xformers>=0.0.26.post1 (from axolotl)
  Downloading xformers-0.0.27.post2-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.0 kB)
Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from fire->axolotl) (1.16.0)
Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from fire->axolotl) (2.4.0)
Collecting markdown2[all] (from fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading markdown2-2.5.0-py2.py3-none-any.whl.metadata (2.2 kB)
Collecting nh3 (from fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading nh3-0.2.18-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB)
Requirement already satisfied: prompt-toolkit>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl) (3.0.47)
Requirement already satisfied: rich>=10.0.0 in /usr/local/lib/python3.10/dist-packages (from fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl) (13.8.1)
Collecting shortuuid (from fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading shortuuid-1.0.13-py3-none-any.whl.metadata (5.8 kB)
Collecting tiktoken (from fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba->axolotl) (0.43.0)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (1.4.0)
Requirement already satisfied: grpcio>=1.48.2 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (1.64.1)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (3.7)
Requirement already satisfied: protobuf!=4.24.0,<5.0.0,>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (3.20.3)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (71.0.4)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl) (3.0.4)
Requirement already satisfied: click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb->axolotl) (8.1.7)
Collecting docker-pycreds>=0.4.0 (from wandb->axolotl)
  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting gitpython!=3.1.29,>=1.0.0 (from wandb->axolotl)
  Downloading GitPython-3.1.43-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: platformdirs in /usr/local/lib/python3.10/dist-packages (from wandb->axolotl) (4.3.2)
Collecting sentry-sdk>=1.0.0 (from wandb->axolotl)
  Downloading sentry_sdk-2.14.0-py2.py3-none-any.whl.metadata (9.7 kB)
Collecting setproctitle (from wandb->axolotl)
  Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.9 kB)
Collecting botocore<1.35.17,>=1.35.16 (from aiobotocore<3.0.0,>=2.5.4->s3fs>=2024.5.0->axolotl)
  Downloading botocore-1.35.16-py3-none-any.whl.metadata (5.7 kB)
Requirement already satisfied: wrapt<2.0.0,>=1.10.10 in /usr/local/lib/python3.10/dist-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs>=2024.5.0->axolotl) (1.16.0)
Collecting aioitertools<1.0.0,>=0.5.1 (from aiobotocore<3.0.0,>=2.5.4->s3fs>=2024.5.0->axolotl)
  Downloading aioitertools-0.12.0-py3-none-any.whl.metadata (3.8 kB)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (2.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (24.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (6.1.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (1.11.1)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.20.0->axolotl) (4.0.3)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.50.2->axolotl) (0.4)
Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.50.2->axolotl) (4.23.0)
Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.50.2->axolotl) (0.12.1)
Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.29,>=1.0.0->wandb->axolotl)
  Downloading gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.2->gcsfs>=2024.5.0->axolotl) (5.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.2->gcsfs>=2024.5.0->axolotl) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.2->gcsfs>=2024.5.0->axolotl) (4.9)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.50.2->axolotl) (1.3.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.50.2->axolotl) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.50.2->axolotl) (4.53.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.50.2->axolotl) (1.4.7)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.50.2->axolotl) (3.1.4)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.50.2->axolotl) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets==2.20.0->axolotl) (2024.1)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets==2.20.0->axolotl) (2024.1)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.10/dist-packages (from prompt-toolkit>=3.0.0->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl) (0.2.13)
Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.0.0->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.0.0->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl) (2.16.1)
Requirement already satisfied: docstring-parser>=0.16 in /usr/local/lib/python3.10/dist-packages (from tyro>=0.5.11->trl==0.9.6->axolotl) (0.16)
Collecting shtab>=1.5.6 (from tyro>=0.5.11->trl==0.9.6->axolotl)
  Downloading shtab-1.7.1-py3-none-any.whl.metadata (7.3 kB)
Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio==3.50.2->axolotl)
  Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
INFO: pip is looking at multiple versions of autoawq-kernels to determine which version is compatible with other requirements. This could take a while.
Collecting autoawq-kernels (from autoawq>=0.2.5->axolotl)
  Downloading autoawq_kernels-0.0.7-cp310-cp310-manylinux2014_x86_64.whl.metadata (2.0 kB)
  Downloading autoawq_kernels-0.0.6-cp310-cp310-manylinux2014_x86_64.whl.metadata (2.0 kB)
Collecting humanfriendly>=9.1 (from coloredlogs->optimum==1.16.2->axolotl)
  Downloading humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)
Collecting starlette<0.39.0,>=0.37.2 (from fastapi->gradio==3.50.2->axolotl)
  Downloading starlette-0.38.5-py3-none-any.whl.metadata (6.0 kB)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib->gcsfs>=2024.5.0->axolotl) (1.3.1)
Requirement already satisfied: google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 in /usr/local/lib/python3.10/dist-packages (from google-cloud-storage->gcsfs>=2024.5.0->axolotl) (2.19.2)
Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-storage->gcsfs>=2024.5.0->axolotl) (2.4.1)
Requirement already satisfied: google-resumable-media>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from google-cloud-storage->gcsfs>=2024.5.0->axolotl) (2.7.2)
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.50.2->axolotl) (3.7.1)
Collecting httpcore==1.* (from httpx->gradio==3.50.2->axolotl)
  Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.50.2->axolotl) (1.3.1)
Collecting wavedrom (from markdown2[all]->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading wavedrom-2.0.3.post3.tar.gz (137 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.7/137.7 kB 14.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting latex2mathml (from markdown2[all]->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading latex2mathml-3.77.0-py3-none-any.whl.metadata (14 kB)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->optimum==1.16.2->axolotl) (1.3.0)
Collecting jmespath<2.0.0,>=0.7.1 (from botocore<1.35.17,>=1.35.16->aiobotocore<3.0.0,>=2.5.4->s3fs>=2024.5.0->axolotl)
  Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.29,>=1.0.0->wandb->axolotl)
  Downloading smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)
Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage->gcsfs>=2024.5.0->axolotl) (1.65.0)
Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in /usr/local/lib/python3.10/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage->gcsfs>=2024.5.0->axolotl) (1.24.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib/python3.10/dist-packages (from google-resumable-media>=2.3.2->google-cloud-storage->gcsfs>=2024.5.0->axolotl) (1.6.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.50.2->axolotl) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.50.2->axolotl) (0.35.1)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.50.2->axolotl) (0.20.0)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich>=10.0.0->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl) (0.1.2)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.2->gcsfs>=2024.5.0->axolotl) (0.6.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib->gcsfs>=2024.5.0->axolotl) (3.2.2)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->gradio==3.50.2->axolotl) (1.2.2)
Collecting svgwrite (from wavedrom->markdown2[all]->fschat@ git+https://github.com/lm-sys/FastChat.git@27a05b04a35510afb1d767ae7e5990cbd278f8fe->axolotl)
  Downloading svgwrite-1.4.3-py3-none-any.whl.metadata (8.8 kB)
Downloading bitsandbytes-0.43.3-py3-none-manylinux_2_24_x86_64.whl (137.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.5/137.5 MB 7.4 MB/s eta 0:00:00
Downloading datasets-2.20.0-py3-none-any.whl (547 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 547.8/547.8 kB 41.7 MB/s eta 0:00:00
Downloading evaluate-0.4.1-py3-none-any.whl (84 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.1/84.1 kB 8.7 MB/s eta 0:00:00
Downloading gradio-3.50.2-py3-none-any.whl (20.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.3/20.3 MB 98.2 MB/s eta 0:00:00
Downloading liger_kernel-0.2.1-py3-none-any.whl (40 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.5/40.5 kB 3.8 MB/s eta 0:00:00
Downloading optimum-1.16.2-py3-none-any.whl (402 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 402.5/402.5 kB 36.5 MB/s eta 0:00:00
Downloading packaging-23.2-py3-none-any.whl (53 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 5.2 MB/s eta 0:00:00
Downloading peft-0.12.0-py3-none-any.whl (296 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 296.4/296.4 kB 27.1 MB/s eta 0:00:00
Downloading pydantic-2.6.3-py3-none-any.whl (395 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 395.2/395.2 kB 31.9 MB/s eta 0:00:00
Downloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)
Downloading scikit_learn-1.4.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.1/12.1 MB 114.5 MB/s eta 0:00:00
Downloading trl-0.9.6-py3-none-any.whl (245 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 245.8/245.8 kB 24.7 MB/s eta 0:00:00
Downloading zstandard-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 112.0 MB/s eta 0:00:00
Downloading gradio_client-0.6.1-py3-none-any.whl (299 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 299.2/299.2 kB 29.4 MB/s eta 0:00:00
Downloading pydantic_core-2.16.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 87.2 MB/s eta 0:00:00
Downloading autoawq-0.2.5-cp310-cp310-manylinux2014_x86_64.whl (84 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.3/84.3 kB 7.8 MB/s eta 0:00:00
Downloading gcsfs-2024.5.0-py2.py3-none-any.whl (34 kB)
Downloading fsspec-2024.5.0-py3-none-any.whl (316 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.1/316.1 kB 30.0 MB/s eta 0:00:00
Downloading s3fs-2024.5.0-py3-none-any.whl (29 kB)
Downloading triton-3.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (209.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.4/209.4 MB 5.2 MB/s eta 0:00:00
Downloading xformers-0.0.27.post2-cp310-cp310-manylinux2014_x86_64.whl (20.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.8/20.8 MB 36.1 MB/s eta 0:00:00
Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)
Downloading art-6.2-py3-none-any.whl (601 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 601.8/601.8 kB 44.3 MB/s eta 0:00:00
Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Downloading hf_transfer-0.1.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 56.2 MB/s eta 0:00:00
Downloading pynvml-11.5.3-py3-none-any.whl (53 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 kB 5.0 MB/s eta 0:00:00
Downloading wandb-0.18.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.5/9.5 MB 119.5 MB/s eta 0:00:00
Downloading aiobotocore-2.15.0-py3-none-any.whl (77 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.4/77.4 kB 8.5 MB/s eta 0:00:00
Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)
Downloading dill-0.3.8-py3-none-any.whl (116 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 kB 12.3 MB/s eta 0:00:00
Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Downloading GitPython-3.1.43-py3-none-any.whl (207 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.3/207.3 kB 20.6 MB/s eta 0:00:00
Downloading orjson-3.10.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.9/141.9 kB 14.9 MB/s eta 0:00:00
Downloading pyarrow-17.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (39.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.9/39.9 MB 13.3 MB/s eta 0:00:00
Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Downloading sentry_sdk-2.14.0-py2.py3-none-any.whl (311 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.4/311.4 kB 31.3 MB/s eta 0:00:00
Downloading tyro-0.8.10-py3-none-any.whl (105 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.7/105.7 kB 11.7 MB/s eta 0:00:00
Downloading uvicorn-0.30.6-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 6.7 MB/s eta 0:00:00
Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.9/129.9 kB 14.2 MB/s eta 0:00:00
Downloading autoawq_kernels-0.0.6-cp310-cp310-manylinux2014_x86_64.whl (33.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 33.4/33.4 MB 59.1 MB/s eta 0:00:00
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 4.0 MB/s eta 0:00:00
Downloading fastapi-0.114.1-py3-none-any.whl (94 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.0/94.0 kB 9.8 MB/s eta 0:00:00
Downloading ffmpy-0.4.0-py3-none-any.whl (5.8 kB)
Downloading httpx-0.27.2-py3-none-any.whl (76 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.4/76.4 kB 7.5 MB/s eta 0:00:00
Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 7.7 MB/s eta 0:00:00
Downloading multiprocess-0.70.16-py310-none-any.whl (134 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.8/134.8 kB 16.3 MB/s eta 0:00:00
Downloading nh3-0.2.18-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (769 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 769.2/769.2 kB 55.1 MB/s eta 0:00:00
Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Downloading python_multipart-0.0.9-py3-none-any.whl (22 kB)
Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)
Downloading shortuuid-1.0.13-py3-none-any.whl (10 kB)
Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 63.6 MB/s eta 0:00:00
Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 194.1/194.1 kB 21.6 MB/s eta 0:00:00
Downloading aioitertools-0.12.0-py3-none-any.whl (24 kB)
Downloading botocore-1.35.16-py3-none-any.whl (12.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.5/12.5 MB 106.3 MB/s eta 0:00:00
Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.7/62.7 kB 6.2 MB/s eta 0:00:00
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 6.3 MB/s eta 0:00:00
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 8.9 MB/s eta 0:00:00
Downloading shtab-1.7.1-py3-none-any.whl (14 kB)
Downloading starlette-0.38.5-py3-none-any.whl (71 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.4/71.4 kB 7.9 MB/s eta 0:00:00
Downloading latex2mathml-3.77.0-py3-none-any.whl (73 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.7/73.7 kB 7.9 MB/s eta 0:00:00
Downloading markdown2-2.5.0-py2.py3-none-any.whl (47 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.2/47.2 kB 5.2 MB/s eta 0:00:00
Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Downloading smmap-5.0.1-py3-none-any.whl (24 kB)
Downloading svgwrite-1.4.3-py3-none-any.whl (67 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.1/67.1 kB 7.1 MB/s eta 0:00:00
Building wheels for collected packages: fire, fschat, wavedrom
  Building wheel for fire (setup.py) ... done
  Created wheel for fire: filename=fire-0.6.0-py2.py3-none-any.whl size=117030 sha256=70b0aa62ddcb134e6277cceac33deccf4650351fab6b0fa0c884eb6f4589ec4c
  Stored in directory: /root/.cache/pip/wheels/d6/6d/5d/5b73fa0f46d01a793713f8859201361e9e581ced8c75e5c6a3
  Building wheel for fschat (pyproject.toml) ... done
  Created wheel for fschat: filename=fschat-0.2.36-py3-none-any.whl size=272080 sha256=85ad46da34f15149fae143f95c159ed13baf5de2790dbf6922e2fcd82e41c81f
  Stored in directory: /root/.cache/pip/wheels/21/dc/55/8647f928ab3e6390d35d3bb898acca851918560726ecdfc42a
  Building wheel for wavedrom (setup.py) ... done
  Created wheel for wavedrom: filename=wavedrom-2.0.3.post3-py2.py3-none-any.whl size=30052 sha256=69c08fad532a0a9ebd2b4f06d81b9f2639c0087658e9d7f06cd5388380a4ef32
  Stored in directory: /root/.cache/pip/wheels/9c/52/8c/38b454b42f712f325e26f633287484c7dc1ad469e1580c5954
Successfully built fire fschat wavedrom
Installing collected packages: pydub, nh3, addict, zstandard, xxhash, websockets, triton, svgwrite, smmap, shtab, shortuuid, setproctitle, sentry-sdk, semantic-version, python-multipart, python-dotenv, pynvml, pydantic-core, pyarrow, packaging, orjson, markdown2, latex2mathml, jmespath, humanfriendly, hf_transfer, h11, fsspec, fire, ffmpy, docker-pycreds, dill, colorama, art, aioitertools, aiofiles, wavedrom, uvicorn, tiktoken, starlette, scikit-learn, responses, pydantic, multiprocess, httpcore, gitdb, coloredlogs, botocore, xformers, tyro, httpx, gitpython, fastapi, bitsandbytes, autoawq-kernels, wandb, gradio-client, fschat, aiobotocore, s3fs, peft, liger-kernel, gradio, datasets, trl, optimum, gcsfs, evaluate, autoawq, axolotl
  Attempting uninstall: pydantic-core
    Found existing installation: pydantic_core 2.23.3
    Uninstalling pydantic_core-2.23.3:
      Successfully uninstalled pydantic_core-2.23.3
  Attempting uninstall: pyarrow
    Found existing installation: pyarrow 14.0.2
    Uninstalling pyarrow-14.0.2:
      Successfully uninstalled pyarrow-14.0.2
  Attempting uninstall: packaging
    Found existing installation: packaging 24.1
    Uninstalling packaging-24.1:
      Successfully uninstalled packaging-24.1
  Attempting uninstall: fsspec
    Found existing installation: fsspec 2024.6.1
    Uninstalling fsspec-2024.6.1:
      Successfully uninstalled fsspec-2024.6.1
  Attempting uninstall: scikit-learn
    Found existing installation: scikit-learn 1.3.2
    Uninstalling scikit-learn-1.3.2:
      Successfully uninstalled scikit-learn-1.3.2
  Attempting uninstall: pydantic
    Found existing installation: pydantic 2.9.1
    Uninstalling pydantic-2.9.1:
      Successfully uninstalled pydantic-2.9.1
  Attempting uninstall: gcsfs
    Found existing installation: gcsfs 2024.6.1
    Uninstalling gcsfs-2024.6.1:
      Successfully uninstalled gcsfs-2024.6.1
  Running setup.py develop for axolotl
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
albumentations 1.4.14 requires pydantic>=2.7.0, but you have pydantic 2.6.3 which is incompatible.
cudf-cu12 24.4.1 requires pyarrow<15.0.0a0,>=14.0.1, but you have pyarrow 17.0.0 which is incompatible.
ibis-framework 8.0.0 requires pyarrow<16,>=2, but you have pyarrow 17.0.0 which is incompatible.
Successfully installed addict-2.4.0 aiobotocore-2.15.0 aiofiles-23.2.1 aioitertools-0.12.0 art-6.2 autoawq-0.2.5 autoawq-kernels-0.0.6 axolotl-0.4.1 bitsandbytes-0.43.3 botocore-1.35.16 colorama-0.4.6 coloredlogs-15.0.1 datasets-2.20.0 dill-0.3.8 docker-pycreds-0.4.0 evaluate-0.4.1 fastapi-0.114.1 ffmpy-0.4.0 fire-0.6.0 fschat-0.2.36 fsspec-2024.5.0 gcsfs-2024.5.0 gitdb-4.0.11 gitpython-3.1.43 gradio-3.50.2 gradio-client-0.6.1 h11-0.14.0 hf_transfer-0.1.8 httpcore-1.0.5 httpx-0.27.2 humanfriendly-10.0 jmespath-1.0.1 latex2mathml-3.77.0 liger-kernel-0.2.1 markdown2-2.5.0 multiprocess-0.70.16 nh3-0.2.18 optimum-1.16.2 orjson-3.10.7 packaging-23.2 peft-0.12.0 pyarrow-17.0.0 pydantic-2.6.3 pydantic-core-2.16.3 pydub-0.25.1 pynvml-11.5.3 python-dotenv-1.0.1 python-multipart-0.0.9 responses-0.18.0 s3fs-2024.5.0 scikit-learn-1.4.2 semantic-version-2.10.0 sentry-sdk-2.14.0 setproctitle-1.3.3 shortuuid-1.0.13 shtab-1.7.1 smmap-5.0.1 starlette-0.38.5 svgwrite-1.4.3 tiktoken-0.7.0 triton-3.0.0 trl-0.9.6 tyro-0.8.10 uvicorn-0.30.6 wandb-0.18.0 wavedrom-2.0.3.post3 websockets-11.0.3 xformers-0.0.27.post2 xxhash-3.5.0 zstandard-0.22.0
Collecting flash-attn==2.5.0
  Downloading flash_attn-2.5.0.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 60.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (2.4.0+cu121)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (0.8.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (23.2)
Collecting ninja (from flash-attn==2.5.0)
  Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl.metadata (5.3 kB)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.16.0)
Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (4.12.2)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (1.13.2)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.1.4)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (2024.5.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->flash-attn==2.5.0) (2.1.5)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->flash-attn==2.5.0) (1.3.0)
Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 26.6 MB/s eta 0:00:00
Building wheels for collected packages: flash-attn

[jackswl]

can confirm this. @winglian could you kindly take a look and update the colab? thanks