unhandled prompt tokenization strategy: sharegpt.load_glaive

[hasan9090]

Please check that this issue hasn't been reported before.

• I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

Hi. I am trying without success so far to preprocess the dataset glaiveai/glaive-function-calling-v2 . At first I used the latest checkout and ran into the problem that the tool role would never get handled which this commit #1779 should address. I don't know why it was not merged and closed but I also think that these changes are needed to make the script recognize the tool role correctly. Even after manually adding these changes from that commit though still does not resolve my issue.

Current behaviour

I get: ValueError: unhandled prompt tokenization strategy: sharegpt.load_glaive . Debugging this, I found that in /src/axolotl/prompt_strategies/init.py it would always in line 28 after returning the func pointer to the sharegpt.load_glaive function , run into "ModuleNotFoundError" which I could not debug. Any ideas what could be happening?

Steps to reproduce

See attached config.yaml.

Config yaml

base_model: NousResearch/Hermes-2-Pro-Mistral-7B
model_type: MistralForCausalLM
tokenizer_type: LlamaTokenizer

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: glaiveai/glaive-function-calling-v2
    type: sharegpt.load_glaive
    conversation: chatml_glaive
    field_human: "human"
    field_model: "gpt"
    field_tool: "tool"
    
    
    roles:
      input: ["USER", "FUNCTION RESPONSE"]
      output: ["ASSISTANT"]

dataset_prepared_path:
#chat_template: chatml
val_set_size: 0.05
output_dir: ./outputs/mistral-out

adapter: lora
lora_model_dir:

sequence_len: 8192
sample_packing: false
pad_to_sequence_len: true

lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:
lora_target_modules:
  - gate_proj
  - down_proj
  - up_proj
  - q_proj
  - v_proj
  - k_proj
  - o_proj

wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 1
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.00002

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: false

loss_watchdog_threshold: 5.0
loss_watchdog_patience: 3

warmup_steps: 10
evals_per_epoch: 4
eval_table_size:
eval_max_new_tokens: 128
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:

Possible solution

No response

Which Operating Systems are you using?

• Linux
• macOS
• Windows

Python Version

3.11.9

axolotl branch-commit

main/6a9cfec

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this bug has not been reported yet.
• I am using the latest version of axolotl.
• I have provided enough information for the maintainers to reproduce and diagnose the issue.