Skip to content

Commit

Permalink
Simplify cache cleanup actions (#365)
Browse files Browse the repository at this point in the history
In our old approach, a workflow file contains a job that uploads the PR
number as an artifact. While the PR is still open, the workflow_run
triggered by it will download the artifact and use the information to clean
up all except the last used cache associated with that original
workflow. When a PR is merged or closed, there will be a post-pr workflow
that uploads the PR number as an artifact and triggers a workflow_run that
clean up all caches associated with the PR. The reason we did it this way
was in the cache cleanup workflows, we did not find an easy way to get the
number of the PR triggering them. This is not convenient because we have to
add jobs uploading artifacts to workflow files.

After some experiments, we have found a reliable way to find the PR number
without using artifacts. The workflow_run's payload always contains the head
SHA of the commit that triggers it, whether the PR comes from a fork or
not. We can then use `gh pr list` to search for that head and obtain the PR
number.
  • Loading branch information
WeiqunZhang committed Sep 5, 2024
1 parent 66b789a commit da2d5a0
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 34 deletions.
15 changes: 0 additions & 15 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,18 +47,3 @@ jobs:
contents: read
security-events: write
uses: ./.github/workflows/codeql.yml

save_pr_number:
if: github.event_name != 'push'
runs-on: ubuntu-latest
steps:
- name: Save PR number
env:
PR_NUMBER: ${{ github.event.number }}
run: |
echo $PR_NUMBER > pr_number.txt
- uses: actions/upload-artifact@v4
with:
name: pr_number
path: pr_number.txt
retention-days: 1
11 changes: 7 additions & 4 deletions .github/workflows/cleanup-cache-postpr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:

jobs:
CleanUpCcacheCachePostPR:
name: Clean Up Ccahe Cache Post PR
name: Clean Up Ccache Cache Post PR
runs-on: ubuntu-latest
permissions:
actions: write
Expand All @@ -17,14 +17,17 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
- name: Clean up ccahe
- name: Clean up ccache
run: |
gh extension install actions/gh-actions-cache
REPO=${{ github.repository }}
gh run download ${{ github.event.workflow_run.id }} -n pr_number
pr_number=`cat pr_number.txt`
# For debugging cat ${GITHUB_EVENT_PATH} to see the payload.
pr_head_sha=${{ github.event.workflow_run.head_sha }}
pr_number=$(gh pr list --state all --search $pr_head_sha --json number --jq '.[0].number')
echo "Post-PR cache cleanup for PR ${pr_number}"
BRANCH=refs/pull/${pr_number}/merge
# Setting this to not fail the workflow while deleting cache keys.
Expand Down
12 changes: 8 additions & 4 deletions .github/workflows/cleanup-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:

jobs:
CleanUpCcacheCache:
name: Clean Up Ccahe Cache for ${{ github.event.workflow_run.name }}
name: Clean Up Ccache Cache for ${{ github.event.workflow_run.name }}
runs-on: ubuntu-latest
permissions:
actions: write
Expand All @@ -17,7 +17,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
- name: Clean up ccahe
- name: Clean up ccache
run: |
gh extension install actions/gh-actions-cache
Expand All @@ -29,9 +29,12 @@ jobs:
# Triggering workflow run name (e.g., LinuxClang)
WORKFLOW_NAME="${{ github.event.workflow_run.name }}"
# For debugging, cat ${GITHUB_EVENT_PATH} to see the payload.
if [[ $EVENT == "pull_request" ]]; then
gh run download ${{ github.event.workflow_run.id }} -n pr_number
pr_number=`cat pr_number.txt`
pr_head_sha=${{ github.event.workflow_run.head_sha }}
pr_number=$(gh pr list --search $pr_head_sha --json number --jq '.[0].number')
echo "Clean up cache for PR ${pr_number}"
BRANCH=refs/pull/${pr_number}/merge
else
BRANCH=refs/heads/${{ github.event.workflow_run.head_branch }}
Expand All @@ -54,6 +57,7 @@ jobs:
IFS=$'\n'
for j in $cached_jobs
do
# Delete all entries except the last used one
old_keys=$(gh actions-cache list -L 100 -R $REPO -B $BRANCH --key "${j}-git-" --sort last-used | cut -f 1 | tail -n +2)
for k in $old_keys
do
Expand Down
18 changes: 7 additions & 11 deletions .github/workflows/post-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,13 @@ on:
types:
- closed

# This workflow does not have the permission to clean up cache for PRs
# originated from a fork. The purpose here is to trigger a workflow_run
# cleanup-cache-postpr.yml that has the right permission.

jobs:
cleanup:
noop:
runs-on: ubuntu-latest
steps:
- name: Save PR number
env:
PR_NUMBER: ${{ github.event.number }}
run: |
echo $PR_NUMBER > pr_number.txt
- uses: actions/upload-artifact@v4
with:
name: pr_number
path: pr_number.txt
retention-days: 1
- name: No OP
run: echo "This workflow is going to trigger CleanUpCachePostPR."

0 comments on commit da2d5a0

Please sign in to comment.