Merge branch 'develop' into migrate-to-sbc

pybamm-team · Sep 2, 2024 · f2fa9f3 · f2fa9f3
2 parents 92fbe27 + 1ab27d1
commit f2fa9f3
Show file tree

Hide file tree

Showing 6 changed files with 76 additions and 125 deletions.
diff --git a/.github/release_workflow.md b/.github/release_workflow.md
@@ -1,10 +1,13 @@
 # Release workflow
 
-This file contains the workflow required to make a `PyBaMM` release on GitHub, PyPI, and conda-forge by the maintainers.
+This file contains the workflow required to make a `PyBaMM` release on
+GitHub, PyPI, and conda-forge by the maintainers.
 
-## rc0 releases (automated)
+## Initial release (automated)
 
-1. The `update_version.yml` workflow will run on every 1st of January, May and September, updating incrementing the version to `vYY.MMrc0` by running `scripts/update_version.py` in the following files -
+1. The `update_version.yml` workflow will run on every 1st of January, May
+   and September, updating incrementing the version to `vYY.MM.0` by running
+   `scripts/update_version.py` in the following files:
 
    - `pybamm/version.py`
    - `docs/conf.py`
@@ -13,21 +16,27 @@ This file contains the workflow required to make a `PyBaMM` release on GitHub, P
    - `vcpkg.json`
    - `CHANGELOG.md`
 
-      These changes will be automatically pushed to a new branch `vYY.MM` and a PR from `vvYY.MM` to `develop` will be created (to sync the branches).
+   These changes will be automatically pushed to a new branch `vYY.MM`
+   and a PR from `vYY.MM` to `main` will be created.
 
-2. Create a new GitHub _pre-release_ with the tag `vYY.MMrc0` from the `vYY.MM` branch and a description copied from `CHANGELOG.md`.
+2. Create a new GitHub _release_ with the tag `vYY.MM.0` from the `vYY.MM`
+   branch and a description copied from `CHANGELOG.md`.
 
-3. This release will automatically trigger `publish_pypi.yml` and create a _pre-release_ on PyPI.
+3. This release will automatically trigger `publish_pypi.yml` and create a
+   _release_ on PyPI.
 
-## rcX releases (manual)
+## Bug fix releases (manual)
 
-If a new release candidate is required after the release of `rc{X-1}` -
+If a new release is required after the release of `vYY.MM.{x-1}` -
 
-1. Cherry-pick the bug fix (no new features should be added to `vYY.MM` once `rc{X-1}` is released) commit to `vYY.MM` branch once the fix is merged into `develop`. The CHANGELOG entry for such fixes should go under the `rc{X-1}` heading in `CHANGELOG.md`
+1. Create a new branch for the `vYY.MM.x` release using the `vYY.MM.{x-1}` tag.
 
-2. Run `update_version.yml` manually while using `append_to_tag` to specify the release candidate version number (`rc1`, `rc2`, ...).
+2. Cherry-pick the bug fixes to `vYY.MM.x` branch once the fix is
+   merged into `develop`. The CHANGELOG entry for such fixes should go under the
+   `YY.MM.x` heading in `CHANGELOG.md`
 
-3. This will increment the version to `vYY.MMrcX` by running `scripts/update_version.py` in the following files -
+3. Run `scripts/update_version.py` manually while setting `VERSION=vYY.MM.x`
+   in your environment. This will update the version in the following files:
 
    - `pybamm/version.py`
    - `docs/conf.py`
@@ -36,45 +45,41 @@ If a new release candidate is required after the release of `rc{X-1}` -
    - `vcpkg.json`
    - `CHANGELOG.md`
 
-      These changes will be automatically pushed to the existing `vYY.MM` branch and a PR will be created to update version strings in `develop`.
+   Commit the changes to your release branch.
 
-4. Create a new GitHub _pre-release_ with the same tag (`vYY.MMrcX`) from the `vYY.MM` branch and a description copied from `CHANGELOG.md`.
+4. Create a PR for the release and configure it to merge into the `main` branch.
 
-5. This release will automatically trigger `publish_pypi.yml` and create a _pre-release_ on PyPI.
-
-## Actual release (manual)
-
-Once satisfied with the release candidates -
-
-1. Run `update_version.yml` manually, leaving the `append_to_tag` field blank ("") for an actual release.
-
-2. This will increment the version to `vYY.MMrcX` by running `scripts/update_version.py` in the following files -
-
-   - `pybamm/version.py`
-   - `docs/conf.py`
-   - `CITATION.cff`
-   - `pyproject.toml`
-   - `vcpkg.json`
-   - `CHANGELOG.md`
-
-      These changes will be automatically pushed to the existing `vYY.MM` branch and a PR will be created to update version strings in `develop`.
-
-3. Next, a PR from `vYY.MM` to `main` will be generated that should be merged once all the tests pass.
-
-4. Create a new GitHub _release_ with the same tag from the `main` branch and a description copied from `CHANGELOG.md`.
-
-5. This release will automatically trigger `publish_pypi.yml` and create a _release_ on PyPI.
+5. Create a new GitHub release with the same tag (`YY.MM.x`) from the `main`
+   branch and a description copied from `CHANGELOG.md`. This release will
+   automatically trigger `publish_pypi.yml` and create a _release_ on PyPI.
 
 ## Other checks
 
 Some other essential things to check throughout the release process -
 
-- If updating our custom vcpkg registory entries [pybamm-team/sundials-vcpkg-registry](https://github.com/pybamm-team/sundials-vcpkg-registry) or [pybamm-team/casadi-vcpkg-registry](https://github.com/pybamm-team/casadi-vcpkg-registry) (used to build Windows wheels), make sure to update the baseline of the registories in vcpkg-configuration.json to the latest commit id.
-- Update jax and jaxlib to the latest version in `pybamm.util` and `pyproject.toml`, fixing any bugs that arise
-- As the release workflow is initiated by the `release` event, it's important to note that the default `GITHUB_REF` used by `actions/checkout` during the checkout process will correspond to the tag created during the release process. Consequently, the workflows will consistently build PyBaMM based on the commit associated with this tag. Should new commits be introduced to the `vYY.MM` branch, such as those addressing build issues, it becomes necessary to manually update this tag to point to the most recent commit -
-  ```
-  git tag -f <tag_name> <commit_hash>
-  git push -f <pybamm-team/PyBaMM_remote_name> <tag_name>  # can only be carried out by the maintainers
-  ```
-- If changes are made to the API, console scripts, entry points, new optional dependencies are added, support for major Python versions is dropped or added, or core project information and metadata are modified at the time of the release, make sure to update the `meta.yaml` file in the `recipe/` folder of the [conda-forge/pybamm-feedstock](https://github.com/conda-forge/pybamm-feedstock) repository accordingly by following the instructions in the [conda-forge documentation](https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-feedstock-repository) and re-rendering the recipe
-- The conda-forge release workflow will automatically be triggered following a stable PyPI release, and the aforementioned updates should be carried out directly in the main repository by pushing changes to the automated PR created by the conda-forge-bot. A manual PR can also be created if the updates are not included in the automated PR for some reason. This manual PR **must** bump the build number in `meta.yaml` and **must** be from a personal fork of the repository.
+- If updating our custom vcpkg registry entries
+  [sundials-vcpkg-registry][SUNDIALS_VCPKG]
+  or [casadi-vcpkg-registry][CASADI_VCPKG] (used to build Windows
+  wheels), make sure to update the baseline of the registries in
+  vcpkg-configuration.json to the latest commit id.
+- Update jax and jaxlib to the latest version in `pybamm.util` and
+  `pyproject.toml`, fixing any bugs that arise.
+- If changes are made to the API, console scripts, entry points, new optional
+  dependencies are added, support for major Python versions is dropped or
+  added, or core project information and metadata are modified at the time
+  of the release, make sure to update the `meta.yaml` file in the `recipe/`
+  folder of the [pybamm-feedstock][PYBAMM_FEED] repository accordingly by
+  following the instructions in the [conda-forge documentation][FEED_GUIDE] and
+  re-rendering the recipe.
+- The conda-forge release workflow will automatically be triggered following
+  a stable PyPI release, and the aforementioned updates should be carried
+  out directly in the main repository by pushing changes to the automated PR
+  created by the conda-forge-bot. A manual PR can also be created if the
+  updates are not included in the automated PR for some reason. This manual
+  PR **must** bump the build number in `meta.yaml` and **must** be from a
+  personal fork of the repository.
+
+[SUNDIALS_VCPKG]: https://github.com/pybamm-team/sundials-vcpkg-registry
+[CASADI_VCPKG]: https://github.com/pybamm-team/casadi-vcpkg-registry
+[PYBAMM_FEED]: https://github.com/conda-forge/pybamm-feedstock
+[FEED_GUIDE]: https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-feedstock-repository
diff --git a/.github/workflows/publish_pypi.yml b/.github/workflows/publish_pypi.yml
@@ -85,8 +85,10 @@ jobs:
             CMAKE_GENERATOR_PLATFORM=x64
             CMAKE_BUILD_PARALLEL_LEVEL=${{ steps.get_num_cores.outputs.count }}
           CIBW_ARCHS: AMD64
-          CIBW_BEFORE_BUILD: python -m pip install scikit-build-core delvewheel pybind11
-          CIBW_REPAIR_WHEEL_COMMAND: delvewheel repair -w {dest_dir} {wheel}
+          CIBW_BEFORE_BUILD: python -m pip install scikit-build-core delvewheel pybind11 # skip CasADi and CMake
+          # Fix access violation because GHA runners have modified PATH that picks wrong
+          # msvcp140.dll, see https://github.com/adang1345/delvewheel/issues/54
+          CIBW_REPAIR_WHEEL_COMMAND: delvewheel repair --add-path C:/Windows/System32 -w {dest_dir} {wheel}
           CIBW_TEST_EXTRAS: "all,dev,jax"
           CIBW_TEST_COMMAND: |
             python -c "import pybamm; print(pybamm.IDAKLUSolver())"
@@ -150,8 +152,8 @@ jobs:
         with:
           python-version: '3.11'
 
-      - name: Set macOS-specific environment variables
-        run: echo "MACOSX_DEPLOYMENT_TARGET=11.0" >> $GITHUB_ENV
+      - name: Install cibuildwheel
+        run: python -m pip install cibuildwheel
 
       - name: Build wheels on macOS
         uses: pypa/cibuildwheel@v2.19.2
@@ -242,14 +244,18 @@ jobs:
             export LDFLAGS="$LDFLAGS -L$PREFIX/lib -lomp"
 
             python scripts/install_KLU_Sundials.py
+            python -m cibuildwheel --output-dir wheelhouse
+        env:
+          # 10.13 for Intel (macos-12/macos-13), 11.0 for Apple Silicon (macos-14 and macos-latest)
+          MACOSX_DEPLOYMENT_TARGET: ${{ matrix.os == 'macos-14' && '11.0' || '10.13' }}
           CIBW_ARCHS_MACOS: auto
           CIBW_BEFORE_BUILD: |
             python -m pip install cmake casadi scikit-build-core delocate pybind11
           CIBW_REPAIR_WHEEL_COMMAND: |
             if [[ $(uname -m) == "x86_64" ]]; then
               delocate-listdeps {wheel} && delocate-wheel -v -w {dest_dir} {wheel}
             elif [[ $(uname -m) == "arm64" ]]; then
-              # Use higher macOS target for now: https://github.com/casadi/casadi/issues/3698
+              # Use higher macOS target for now since casadi/libc++.1.0.dylib is still not fixed
               delocate-listdeps {wheel} && delocate-wheel -v -w {dest_dir} {wheel} --require-target-macos-version 11.1
               for file in {dest_dir}/*.whl; do mv "$file" "${file//macosx_11_1/macosx_11_0}"; done
             fi

diff --git a/.github/workflows/update_version.yml b/.github/workflows/update_version.yml
@@ -1,11 +1,6 @@
 name: Update version
 
 on:
-  workflow_dispatch:
-    inputs:
-      append_to_tag:
-        description: 'Leave blank for an actual release or "rc1", "rc2", ..., for release candidates."'
-        default: ""
   schedule:
      # Run at 10 am UTC on day-of-month 1 in January, May, and September.
      - cron: "0 10 1 1,5,9 *"
@@ -18,94 +13,39 @@ jobs:
 
     steps:
       - name: Get current date for the first release candidate
-        if: github.event_name == 'schedule'
         run: |
-          echo "VERSION=$(date +'v%y.%-m')rc0" >> $GITHUB_ENV
-          echo "NON_RC_VERSION=$(date +'v%y.%-m')" >> $GITHUB_ENV
+          echo "VERSION=$(date +'v%y.%-m').0" >> $GITHUB_ENV
 
-      - name: Get current date for a manual release
-        if: github.event_name == 'workflow_dispatch'
-        run: |
-          echo "VERSION=$(date +'v%y.%-m')${{ github.event.inputs.append_to_tag }}" >> $GITHUB_ENV
-          echo "NON_RC_VERSION=$(date +'v%y.%-m')" >> $GITHUB_ENV
-
-      # the schedule workflow is for rc0 release
       - uses: actions/checkout@v4
-        if: github.event_name == 'schedule'
         with:
           ref: 'develop'
 
-      # the dispatch workflow is for rcX and final releases
-      - uses: actions/checkout@v4
-        if: github.event_name == 'workflow_dispatch'
-        with:
-          ref: '${{ env.NON_RC_VERSION }}'
-
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
           python-version: 3.12
 
       - name: Install dependencies
         run: |
-          pip install --editable ".[all]"
+          pip install -e ".[all]"
 
-      # update all the version strings and add CHANGELOG headings
+      # Update all the version strings and add CHANGELOG headings
       - name: Update version
         run: python scripts/update_version.py
 
-      # create a new version branch for rc0 release and commit
+      # Create a new version branch for the release and commit
       - uses: EndBug/add-and-commit@v9
-        if: github.event_name == 'schedule'
         with:
           message: 'Bump to ${{ env.VERSION }}'
-          new_branch: '${{ env.NON_RC_VERSION }}'
-
-      # use the already created release branch for rcX + final releases
-      # and commit
-      - uses: EndBug/add-and-commit@v9
-        if: github.event_name == 'workflow_dispatch'
-        with:
-          message: 'Bump to ${{ env.VERSION }}'
-
-      # checkout to develop for updating versions in the same
-      - uses: actions/checkout@v4
-        with:
-          ref: 'develop'
-
-      # update all the version strings
-      - name: Update version
-        if: github.event_name == 'workflow_dispatch'
-        run: python scripts/update_version.py
-
-      # create a pull request updating versions in develop
-      - name: Create Pull Request
-        id: version_pr
-        uses: peter-evans/create-pull-request@v6
-        with:
-          delete-branch: true
-          branch-suffix: short-commit-hash
-          base: develop
-          commit-message: Update version to ${{ env.VERSION }}
-          title: Bump to ${{ env.VERSION }}
-          body: |
-            - [x] Update to ${{ env.VERSION }}
-            - [ ] Check the [release workflow](https://github.com/pybamm-team/PyBaMM/blob/develop/.github/release_workflow.md)
-
-      # checkout to the version branch for the final release
-      - uses: actions/checkout@v4
-        if: github.event_name == 'workflow_dispatch' && !startsWith(github.event.inputs.append_to_tag, 'rc')
-        with:
-          ref: '${{ env.NON_RC_VERSION }}'
+          new_branch: '${{ env.VERSION }}'
 
-      # for final releases, create a PR from version branch to main
-      - name: Make a PR from ${{ env.NON_RC_VERSION }} to main
+      # Create a PR from version branch to main
+      - name: Make a PR from ${{ env.VERSION }} to main
         id: release_pr
-        if: github.event_name == 'workflow_dispatch' && !startsWith(github.event.inputs.append_to_tag, 'rc')
         uses: repo-sync/pull-request@v2
         with:
-          source_branch: '${{ env.NON_RC_VERSION }}'
+          source_branch: '${{ env.VERSION }}'
           destination_branch: "main"
-          pr_title: "Make release ${{ env.NON_RC_VERSION }}"
+          pr_title: "Make release ${{ env.VERSION }}"
           pr_body: "**Check the [release workflow](https://github.com/pybamm-team/PyBaMM/blob/develop/.github/release_workflow.md)**"
           github_token: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -4,7 +4,7 @@ ci:
 
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: "v0.6.2"
+    rev: "v0.6.3"
     hooks:
       - id: ruff
         args: [--fix, --show-fixes]

diff --git a/pyproject.toml b/pyproject.toml
@@ -2,9 +2,9 @@
 requires = [
     "scikit-build-core>=0.10",
     # On Windows, use the CasADi vcpkg registry and CMake bundled from MSVC
-    "casadi>=3.6.5; platform_system!='Windows'",
+    "casadi>=3.6.6; platform_system!='Windows'",
     # Note: the version of CasADi as a build-time dependency should be matched
-    # cross platforms, so updates to its minimum version here should be accompanied
+    # across platforms, so updates to its minimum version here should be accompanied
     # by a version bump in https://github.com/pybamm-team/casadi-vcpkg-registry.
     "cmake>=3.13; platform_system!='Windows'",
     "pybind11>=2.13.4",
@@ -37,7 +37,7 @@ classifiers = [
 dependencies = [
     "numpy>=1.23.5,<2.0.0",
     "scipy>=1.11.4",
-    "casadi>=3.6.5",
+    "casadi>=3.6.6",
     "xarray>=2022.6.0",
     "anytree>=2.8.0",
     "sympy>=1.12",

diff --git a/vcpkg-configuration.json b/vcpkg-configuration.json
@@ -13,7 +13,7 @@
     {
       "kind": "git",
       "repository": "https://github.com/pybamm-team/casadi-vcpkg-registry.git",
-      "baseline": "ceee3ed50246744cdef43517d7d7617b8ac291e7",
+      "baseline": "1cb93f2fb71be26c874db724940ef8e604ee558e",
       "packages": ["casadi"]
     }
   ]