Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BE][1/n] simplify train.py and rename things for consistency #494

Merged
merged 3 commits into from
Aug 1, 2024
Merged

[BE][1/n] simplify train.py and rename things for consistency #494

merged 3 commits into from
Aug 1, 2024

Commits on Jul 31, 2024

  1. [BE][1/n] simplify train.py 

    [ghstack-poisoned]
    @tianyu-l
    tianyu-l committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c71bdce View commit details
    Browse the repository at this point in the history

  2. Update on "[BE][1/n] simplify train.py and rename things for consiste… 

    …ncy"


Including:
- move TrainState from `train.py` to `checkpoint.py`
- create `optimizer.py` to include things related to optimizers and lr scheduler.
- rename `logging_utils.py` to `logging.py`
- unify various build prefixes (`build_`, `create_`, `get_`) to `build_`
- simplify utils import by doing `import torchtitan.utils as utils`
- move `get_metrics_rank` from `utils.py` to `metrics.py` to make `build_metric_logger` simpler
- create `GarbageCollection` in `utils.py` to hide gc details from `train.py`
- reorder definition and initialization of some objects in `train.py` to be closer to where they are first used
- expose `build_pipeline_schedule` to `torchtitan.parallelisms`
- other minor improvements to reduce the amount of import in `train.py`

After this refactoring, LoC for import in `train.py` drops from 51 to 23.

[ghstack-poisoned]
    @tianyu-l
    tianyu-l committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    0566d80 View commit details
    Browse the repository at this point in the history

  3. Update on "[BE][1/n] simplify train.py and rename things for consiste… 

    …ncy"


Including:
- move TrainState from `train.py` to `checkpoint.py`
- create `optimizer.py` to include things related to optimizers and lr scheduler.
- rename `logging_utils.py` to `logging.py`
- unify various build prefixes (`build_`, `create_`, `get_`) to `build_`
- simplify utils import by doing `import torchtitan.utils as utils`
- move `get_metrics_rank` from `utils.py` to `metrics.py` to make `build_metric_logger` simpler
- create `GarbageCollection` in `utils.py` to hide gc details from `train.py`
- reorder definition and initialization of some objects in `train.py` to be closer to where they are first used
- expose `build_pipeline_schedule` to `torchtitan.parallelisms`
- other minor improvements to reduce the amount of import in `train.py`

After this refactoring, LoC for import in `train.py` drops from 51 to 23.

[ghstack-poisoned]
    @tianyu-l
    tianyu-l committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    e35f66c View commit details
    Browse the repository at this point in the history