Skip to content

Commit

Permalink
Streamline release process (pybamm-team#4376)
Browse files Browse the repository at this point in the history 
* Modify update version workflow

* Simplify workflow

* A few more fixes

* Style

* Fix link errors

* Remove manual trigger

* Remove redundant work

* Fix workflow text

* Test workflow

* Change target branch

* Fix

* Remove some test code
  • Loading branch information
@kratman
kratman authored Sep 2, 2024
1 parent 16e06f5 commit 4b4b98b
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 116 deletions.
97 changes: 51 additions & 46 deletions .github/release_workflow.md
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# Release workflow

This file contains the workflow required to make a `PyBaMM` release on GitHub, PyPI, and conda-forge by the maintainers.
This file contains the workflow required to make a `PyBaMM` release on
GitHub, PyPI, and conda-forge by the maintainers.

## rc0 releases (automated)
## Initial release (automated)

1. The `update_version.yml` workflow will run on every 1st of January, May and September, updating incrementing the version to `vYY.MMrc0` by running `scripts/update_version.py` in the following files -
1. The `update_version.yml` workflow will run on every 1st of January, May
and September, updating incrementing the version to `vYY.MM.0` by running
`scripts/update_version.py` in the following files:

- `pybamm/version.py`
- `docs/conf.py`
Expand All @@ -13,21 +16,27 @@ This file contains the workflow required to make a `PyBaMM` release on GitHub, P
- `vcpkg.json`
- `CHANGELOG.md`

These changes will be automatically pushed to a new branch `vYY.MM` and a PR from `vvYY.MM` to `develop` will be created (to sync the branches).
These changes will be automatically pushed to a new branch `vYY.MM`
and a PR from `vYY.MM` to `main` will be created.

2. Create a new GitHub _pre-release_ with the tag `vYY.MMrc0` from the `vYY.MM` branch and a description copied from `CHANGELOG.md`.
2. Create a new GitHub _release_ with the tag `vYY.MM.0` from the `vYY.MM`
branch and a description copied from `CHANGELOG.md`.

3. This release will automatically trigger `publish_pypi.yml` and create a _pre-release_ on PyPI.
3. This release will automatically trigger `publish_pypi.yml` and create a
_release_ on PyPI.

## rcX releases (manual)
## Bug fix releases (manual)

If a new release candidate is required after the release of `rc{X-1}` -
If a new release is required after the release of `vYY.MM.{x-1}` -

1. Cherry-pick the bug fix (no new features should be added to `vYY.MM` once `rc{X-1}` is released) commit to `vYY.MM` branch once the fix is merged into `develop`. The CHANGELOG entry for such fixes should go under the `rc{X-1}` heading in `CHANGELOG.md`
1. Create a new branch for the `vYY.MM.x` release using the `vYY.MM.{x-1}` tag.

2. Run `update_version.yml` manually while using `append_to_tag` to specify the release candidate version number (`rc1`, `rc2`, ...).
2. Cherry-pick the bug fixes to `vYY.MM.x` branch once the fix is
merged into `develop`. The CHANGELOG entry for such fixes should go under the
`YY.MM.x` heading in `CHANGELOG.md`

3. This will increment the version to `vYY.MMrcX` by running `scripts/update_version.py` in the following files -
3. Run `scripts/update_version.py` manually while setting `VERSION=vYY.MM.x`
in your environment. This will update the version in the following files:

- `pybamm/version.py`
- `docs/conf.py`
Expand All @@ -36,45 +45,41 @@ If a new release candidate is required after the release of `rc{X-1}` -
- `vcpkg.json`
- `CHANGELOG.md`

These changes will be automatically pushed to the existing `vYY.MM` branch and a PR will be created to update version strings in `develop`.
Commit the changes to your release branch.

4. Create a new GitHub _pre-release_ with the same tag (`vYY.MMrcX`) from the `vYY.MM` branch and a description copied from `CHANGELOG.md`.
4. Create a PR for the release and configure it to merge into the `main` branch.

5. This release will automatically trigger `publish_pypi.yml` and create a _pre-release_ on PyPI.

## Actual release (manual)

Once satisfied with the release candidates -

1. Run `update_version.yml` manually, leaving the `append_to_tag` field blank ("") for an actual release.

2. This will increment the version to `vYY.MMrcX` by running `scripts/update_version.py` in the following files -

- `pybamm/version.py`
- `docs/conf.py`
- `CITATION.cff`
- `pyproject.toml`
- `vcpkg.json`
- `CHANGELOG.md`

These changes will be automatically pushed to the existing `vYY.MM` branch and a PR will be created to update version strings in `develop`.

3. Next, a PR from `vYY.MM` to `main` will be generated that should be merged once all the tests pass.

4. Create a new GitHub _release_ with the same tag from the `main` branch and a description copied from `CHANGELOG.md`.

5. This release will automatically trigger `publish_pypi.yml` and create a _release_ on PyPI.
5. Create a new GitHub release with the same tag (`YY.MM.x`) from the `main`
branch and a description copied from `CHANGELOG.md`. This release will
automatically trigger `publish_pypi.yml` and create a _release_ on PyPI.

## Other checks

Some other essential things to check throughout the release process -

- If updating our custom vcpkg registory entries [pybamm-team/sundials-vcpkg-registry](https://github.com/pybamm-team/sundials-vcpkg-registry) or [pybamm-team/casadi-vcpkg-registry](https://github.com/pybamm-team/casadi-vcpkg-registry) (used to build Windows wheels), make sure to update the baseline of the registories in vcpkg-configuration.json to the latest commit id.
- Update jax and jaxlib to the latest version in `pybamm.util` and `pyproject.toml`, fixing any bugs that arise
- As the release workflow is initiated by the `release` event, it's important to note that the default `GITHUB_REF` used by `actions/checkout` during the checkout process will correspond to the tag created during the release process. Consequently, the workflows will consistently build PyBaMM based on the commit associated with this tag. Should new commits be introduced to the `vYY.MM` branch, such as those addressing build issues, it becomes necessary to manually update this tag to point to the most recent commit -
```
git tag -f <tag_name> <commit_hash>
git push -f <pybamm-team/PyBaMM_remote_name> <tag_name> # can only be carried out by the maintainers
```
- If changes are made to the API, console scripts, entry points, new optional dependencies are added, support for major Python versions is dropped or added, or core project information and metadata are modified at the time of the release, make sure to update the `meta.yaml` file in the `recipe/` folder of the [conda-forge/pybamm-feedstock](https://github.com/conda-forge/pybamm-feedstock) repository accordingly by following the instructions in the [conda-forge documentation](https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-feedstock-repository) and re-rendering the recipe
- The conda-forge release workflow will automatically be triggered following a stable PyPI release, and the aforementioned updates should be carried out directly in the main repository by pushing changes to the automated PR created by the conda-forge-bot. A manual PR can also be created if the updates are not included in the automated PR for some reason. This manual PR **must** bump the build number in `meta.yaml` and **must** be from a personal fork of the repository.
- If updating our custom vcpkg registry entries
[sundials-vcpkg-registry][SUNDIALS_VCPKG]
or [casadi-vcpkg-registry][CASADI_VCPKG] (used to build Windows
wheels), make sure to update the baseline of the registries in
vcpkg-configuration.json to the latest commit id.
- Update jax and jaxlib to the latest version in `pybamm.util` and
`pyproject.toml`, fixing any bugs that arise.
- If changes are made to the API, console scripts, entry points, new optional
dependencies are added, support for major Python versions is dropped or
added, or core project information and metadata are modified at the time
of the release, make sure to update the `meta.yaml` file in the `recipe/`
folder of the [pybamm-feedstock][PYBAMM_FEED] repository accordingly by
following the instructions in the [conda-forge documentation][FEED_GUIDE] and
re-rendering the recipe.
- The conda-forge release workflow will automatically be triggered following
a stable PyPI release, and the aforementioned updates should be carried
out directly in the main repository by pushing changes to the automated PR
created by the conda-forge-bot. A manual PR can also be created if the
updates are not included in the automated PR for some reason. This manual
PR **must** bump the build number in `meta.yaml` and **must** be from a
personal fork of the repository.

[SUNDIALS_VCPKG]: https://github.com/pybamm-team/sundials-vcpkg-registry
[CASADI_VCPKG]: https://github.com/pybamm-team/casadi-vcpkg-registry
[PYBAMM_FEED]: https://github.com/conda-forge/pybamm-feedstock
[FEED_GUIDE]: https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-feedstock-repository
79 changes: 9 additions & 70 deletions .github/workflows/update_version.yml
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
name: Update version

on:
workflow_dispatch:
inputs:
append_to_tag:
description: 'Leave blank for an actual release or "rc1", "rc2", ..., for release candidates."'
default: ""
schedule:
# Run at 10 am UTC on day-of-month 1 in January, May, and September.
- cron: "0 10 1 1,5,9 *"
Expand All @@ -18,95 +13,39 @@ jobs:

steps:
- name: Get current date for the first release candidate
if: github.event_name == 'schedule'
run: |
echo "VERSION=$(date +'v%y.%-m')rc0" >> $GITHUB_ENV
echo "NON_RC_VERSION=$(date +'v%y.%-m')" >> $GITHUB_ENV
echo "VERSION=$(date +'v%y.%-m').0" >> $GITHUB_ENV
- name: Get current date for a manual release
if: github.event_name == 'workflow_dispatch'
run: |
echo "VERSION=$(date +'v%y.%-m')${{ github.event.inputs.append_to_tag }}" >> $GITHUB_ENV
echo "NON_RC_VERSION=$(date +'v%y.%-m')" >> $GITHUB_ENV
# the schedule workflow is for rc0 release
- uses: actions/checkout@v4
if: github.event_name == 'schedule'
with:
ref: 'develop'

# the dispatch workflow is for rcX and final releases
- uses: actions/checkout@v4
if: github.event_name == 'workflow_dispatch'
with:
ref: '${{ env.NON_RC_VERSION }}'

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.12

- name: Install dependencies
run: |
pip install wheel
pip install --editable ".[all]"
pip install -e ".[all]"
# update all the version strings and add CHANGELOG headings
# Update all the version strings and add CHANGELOG headings
- name: Update version
run: python scripts/update_version.py

# create a new version branch for rc0 release and commit
# Create a new version branch for the release and commit
- uses: EndBug/add-and-commit@v9
if: github.event_name == 'schedule'
with:
message: 'Bump to ${{ env.VERSION }}'
new_branch: '${{ env.NON_RC_VERSION }}'

# use the already created release branch for rcX + final releases
# and commit
- uses: EndBug/add-and-commit@v9
if: github.event_name == 'workflow_dispatch'
with:
message: 'Bump to ${{ env.VERSION }}'

# checkout to develop for updating versions in the same
- uses: actions/checkout@v4
with:
ref: 'develop'

# update all the version strings
- name: Update version
if: github.event_name == 'workflow_dispatch'
run: python scripts/update_version.py

# create a pull request updating versions in develop
- name: Create Pull Request
id: version_pr
uses: peter-evans/create-pull-request@v6
with:
delete-branch: true
branch-suffix: short-commit-hash
base: develop
commit-message: Update version to ${{ env.VERSION }}
title: Bump to ${{ env.VERSION }}
body: |
- [x] Update to ${{ env.VERSION }}
- [ ] Check the [release workflow](https://github.com/pybamm-team/PyBaMM/blob/develop/.github/release_workflow.md)
# checkout to the version branch for the final release
- uses: actions/checkout@v4
if: github.event_name == 'workflow_dispatch' && !startsWith(github.event.inputs.append_to_tag, 'rc')
with:
ref: '${{ env.NON_RC_VERSION }}'
new_branch: '${{ env.VERSION }}'

# for final releases, create a PR from version branch to main
- name: Make a PR from ${{ env.NON_RC_VERSION }} to main
# Create a PR from version branch to main
- name: Make a PR from ${{ env.VERSION }} to main
id: release_pr
if: github.event_name == 'workflow_dispatch' && !startsWith(github.event.inputs.append_to_tag, 'rc')
uses: repo-sync/pull-request@v2
with:
source_branch: '${{ env.NON_RC_VERSION }}'
source_branch: '${{ env.VERSION }}'
destination_branch: "main"
pr_title: "Make release ${{ env.NON_RC_VERSION }}"
pr_title: "Make release ${{ env.VERSION }}"
pr_body: "**Check the [release workflow](https://github.com/pybamm-team/PyBaMM/blob/develop/.github/release_workflow.md)**"
github_token: ${{ secrets.GITHUB_TOKEN }}

0 comments on commit 4b4b98b

Please sign in to comment.