-
-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build Docker image and push to GHCR #230
base: unstable/v1
Are you sure you want to change the base?
Changes from all commits
eb50ad4
1e0ccc9
6364a07
a6e3290
e236daa
b17295d
9337018
59e3edf
90ff142
61e875d
c962c33
1bcf9d8
80b172c
6eb8d30
8d071a8
1ce7f9d
5521a4f
031df10
314a411
fa8a0e8
580d9d0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
--- | ||
|
||
name: 🏗️ | ||
|
||
on: # yamllint disable-line rule:truthy | ||
pull_request: | ||
push: | ||
branches: ["release/*", "unstable/*"] | ||
workflow_dispatch: | ||
inputs: | ||
tag: | ||
description: Docker image tag | ||
required: true | ||
type: string | ||
|
||
jobs: | ||
build-and-push: | ||
runs-on: ubuntu-latest | ||
timeout-minutes: 10 | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Build Docker image | ||
run: | | ||
DOCKER_TAG="${DOCKER_TAG/'/'/'-'}" | ||
DOCKER_TAG_MAJOR=$(echo "$DOCKER_TAG" | cut -d '.' -f 1) | ||
DOCKER_TAG_MAJOR_MINOR=$(echo "$DOCKER_TAG" | cut -d '.' -f 1-2) | ||
IMAGE="ghcr.io/$GITHUB_REPOSITORY:${DOCKER_TAG}" | ||
IMAGE_MAJOR="ghcr.io/$GITHUB_REPOSITORY:${DOCKER_TAG_MAJOR}" | ||
IMAGE_MAJOR_MINOR="ghcr.io/$GITHUB_REPOSITORY:${DOCKER_TAG_MAJOR_MINOR}" | ||
echo "IMAGE=$IMAGE" >>"$GITHUB_ENV" | ||
echo "IMAGE_MAJOR=$IMAGE_MAJOR" >>"$GITHUB_ENV" | ||
echo "IMAGE_MAJOR_MINOR=$IMAGE_MAJOR_MINOR" >>"$GITHUB_ENV" | ||
docker build . \ | ||
--build-arg BUILDKIT_INLINE_CACHE=1 \ | ||
--cache-from $IMAGE \ | ||
--tag $IMAGE | ||
docker tag $IMAGE $IMAGE_MAJOR | ||
docker tag $IMAGE $IMAGE_MAJOR_MINOR | ||
env: | ||
DOCKER_TAG: ${{ inputs.tag || github.ref_name }} | ||
- name: Log in to GHCR | ||
if: github.event_name != 'pull_request' | ||
run: >- | ||
echo ${{ secrets.GITHUB_TOKEN }} | | ||
webknjaz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
docker login ghcr.io -u $GITHUB_ACTOR --password-stdin | ||
- name: Push Docker image to GHCR | ||
if: github.event_name != 'pull_request' | ||
run: | | ||
docker push $IMAGE | ||
docker push $IMAGE_MAJOR | ||
docker push $IMAGE_MAJOR_MINOR | ||
smoke-test: | ||
br3ndonland marked this conversation as resolved.
Show resolved
Hide resolved
|
||
needs: | ||
- build-and-push | ||
uses: ./.github/workflows/reusable-smoke-test.yml |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,122 @@ | ||
--- | ||
|
||
name: ♻️ 🧪 | ||
|
||
on: # yamllint disable-line rule:truthy | ||
workflow_call: | ||
|
||
env: | ||
devpi-password: abcd1234 | ||
devpi-username: root | ||
devpi-port: 3141 | ||
|
||
FORCE_COLOR: 1 # Request colored output from CLI tools supporting it | ||
MYPY_FORCE_COLOR: 1 # MyPy's color enforcement | ||
PIP_DISABLE_PIP_VERSION_CHECK: 1 | ||
PIP_NO_PYTHON_VERSION_WARNING: 1 | ||
PIP_NO_WARN_SCRIPT_LOCATION: 1 | ||
PY_COLORS: 1 # Recognized by the `py` package, dependency of `pytest` | ||
TOX_PARALLEL_NO_SPINNER: 1 | ||
TOX_TESTENV_PASSENV: >- # Make tox-wrapped tools see color requests | ||
FORCE_COLOR | ||
MYPY_FORCE_COLOR | ||
NO_COLOR | ||
PY_COLORS | ||
PYTEST_THEME | ||
PYTEST_THEME_MODE | ||
|
||
jobs: | ||
fail-fast: | ||
|
||
strategy: | ||
matrix: | ||
os: [macos-latest, windows-latest] | ||
|
||
runs-on: ${{ matrix.os }} | ||
|
||
timeout-minutes: 2 | ||
|
||
steps: | ||
- name: Check out the action locally | ||
uses: actions/checkout@v3 | ||
with: | ||
path: test | ||
- name: Fail-fast in unsupported environments | ||
continue-on-error: true | ||
id: fail-fast | ||
uses: ./test | ||
- name: Error if action did not fail-fast in unsupported environments | ||
if: steps.fail-fast.outcome == 'success' | ||
run: | | ||
>&2 echo This action should fail-fast in unsupported environments. | ||
exit 1 | ||
|
||
smoke-test: | ||
|
||
runs-on: ubuntu-latest | ||
|
||
services: | ||
devpi: | ||
image: muccg/devpi | ||
env: | ||
DEVPI_PASSWORD: ${{ env.devpi-password }} | ||
ports: | ||
- 3141 | ||
|
||
timeout-minutes: 2 | ||
|
||
steps: | ||
- name: Check out the action locally | ||
uses: actions/checkout@v3 | ||
with: | ||
path: test | ||
- name: Install the packaging-related tools | ||
run: python3 -m pip install build twine | ||
env: | ||
PIP_CONSTRAINT: test/requirements/runtime.txt | ||
- name: Create the stub package importable directory | ||
run: mkdir -pv src/test_package | ||
- name: Populate the stub package `__init__.py` | ||
run: echo '__version__ = "0.1"' > src/test_package/__init__.py | ||
- name: Populate the stub package `README.md` | ||
run: echo "# Test Package" > README.md | ||
- name: Populate the stub package `pyproject.toml` | ||
run: echo "$CONTENTS" > pyproject.toml | ||
env: | ||
CONTENTS: | | ||
[build-system] | ||
requires = [ | ||
"setuptools == 65.6.3", | ||
] | ||
build-backend = "setuptools.build_meta" | ||
|
||
[project] | ||
name = "test-package" | ||
version = "0.1" | ||
readme = "README.md" | ||
- name: Build the stub package sdist and wheel distributions | ||
run: python3 -m build | ||
- name: Register the stub package in devpi | ||
run: twine register dist/*.tar.gz | ||
env: | ||
TWINE_USERNAME: ${{ env.devpi-username }} | ||
TWINE_PASSWORD: ${{ env.devpi-password }} | ||
TWINE_REPOSITORY_URL: >- | ||
http://localhost:${{ | ||
job.services.devpi.ports[env.devpi-port] | ||
}}/${{ | ||
env.devpi-username | ||
}}/public/ | ||
- name: ✅ Smoke-test the locally checked out action | ||
uses: ./test | ||
env: | ||
DEBUG: >- | ||
true | ||
PATH: utter-nonsense | ||
with: | ||
user: ${{ env.devpi-username }} | ||
password: ${{ env.devpi-password }} | ||
repository-url: >- | ||
http://devpi:${{ env.devpi-port }}/${{ env.devpi-username }}/public/ | ||
|
||
... |
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -91,15 +91,71 @@ branding: | |||||||||||||
color: yellow | ||||||||||||||
icon: upload-cloud | ||||||||||||||
runs: | ||||||||||||||
using: docker | ||||||||||||||
image: Dockerfile | ||||||||||||||
args: | ||||||||||||||
- ${{ inputs.user }} | ||||||||||||||
- ${{ inputs.password }} | ||||||||||||||
- ${{ inputs.repository-url }} | ||||||||||||||
- ${{ inputs.packages-dir }} | ||||||||||||||
- ${{ inputs.verify-metadata }} | ||||||||||||||
- ${{ inputs.skip-existing }} | ||||||||||||||
- ${{ inputs.verbose }} | ||||||||||||||
- ${{ inputs.print-hash }} | ||||||||||||||
- ${{ inputs.attestations }} | ||||||||||||||
using: composite | ||||||||||||||
steps: | ||||||||||||||
br3ndonland marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
- name: Fail-fast in unsupported environments | ||||||||||||||
if: runner.os != 'Linux' | ||||||||||||||
run: | | ||||||||||||||
>&2 echo This action is only able to run under GNU/Linux environments | ||||||||||||||
exit 1 | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you add CI jobs testing that this fails under corresponding envs? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added CI jobs that test failure on macOS and Windows. |
||||||||||||||
shell: bash -eEuo pipefail {0} | ||||||||||||||
- name: Reset path if needed | ||||||||||||||
run: | | ||||||||||||||
# Reset path if needed | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why would this be needed outside the container? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Because you've set up a test that modifies the gh-action-pypi-publish/.github/workflows/self-smoke-test-action.yml Lines 85 to 90 in 699cd61
|
||||||||||||||
# https://github.com/pypa/gh-action-pypi-publish/issues/112 | ||||||||||||||
if [[ $PATH != *"/usr/bin"* ]]; then | ||||||||||||||
echo "\$PATH=$PATH. Resetting \$PATH for GitHub Actions." | ||||||||||||||
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" | ||||||||||||||
echo "PATH=$PATH" >>"$GITHUB_ENV" | ||||||||||||||
echo "$PATH" >>"$GITHUB_PATH" | ||||||||||||||
echo "\$PATH reset. \$PATH=$PATH" | ||||||||||||||
fi | ||||||||||||||
shell: bash | ||||||||||||||
- name: Set repo and ref from which to run Docker container action | ||||||||||||||
id: set-repo-and-ref | ||||||||||||||
run: | | ||||||||||||||
# Set repo and ref from which to run Docker container action | ||||||||||||||
# to handle cases in which `github.action_` context is not set | ||||||||||||||
# https://github.com/actions/runner/issues/2473 | ||||||||||||||
REF=${{ env.ACTION_REF || env.PR_REF || github.ref_name }} | ||||||||||||||
REPO=${{ env.ACTION_REPO || env.PR_REPO || github.repository }} | ||||||||||||||
REPO_ID=${{ env.PR_REPO_ID || github.repository_id }} | ||||||||||||||
echo "ref=$REF" >>"$GITHUB_OUTPUT" | ||||||||||||||
echo "repo=$REPO" >>"$GITHUB_OUTPUT" | ||||||||||||||
echo "repo-id=$REPO_ID" >>"$GITHUB_OUTPUT" | ||||||||||||||
shell: bash | ||||||||||||||
env: | ||||||||||||||
ACTION_REF: ${{ github.action_ref }} | ||||||||||||||
ACTION_REPO: ${{ github.action_repository }} | ||||||||||||||
PR_REF: ${{ github.event.pull_request.head.ref }} | ||||||||||||||
PR_REPO: ${{ github.event.pull_request.head.repo.full_name }} | ||||||||||||||
PR_REPO_ID: ${{ github.event.pull_request.base.repo.id }} | ||||||||||||||
- name: Check out action repo | ||||||||||||||
uses: actions/checkout@v4 | ||||||||||||||
with: | ||||||||||||||
path: action-repo | ||||||||||||||
ref: ${{ steps.set-repo-and-ref.outputs.ref }} | ||||||||||||||
repository: ${{ steps.set-repo-and-ref.outputs.repo }} | ||||||||||||||
- name: Create Docker container action | ||||||||||||||
run: | | ||||||||||||||
# Create Docker container action | ||||||||||||||
python create-docker-action.py | ||||||||||||||
webknjaz marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry if I'm missing something here: why is creating the container action done dynamically like this? Is there a reason it can't be a static file? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, the reason is that otherwise there's no way to specify the correct Docker tag. Docker actions support pulling in pre-built Docker images by supplying a registry address to the # this works but the image tag can't be customized
runs:
using: docker
image: docker://ghcr.io/pypa/gh-action-pypi-publish:release-v1.8 # this doesn't work because `image:` doesn't support context
runs:
using: docker
image: docker://ghcr.io/pypa/gh-action-pypi-publish:${{ github.action_ref }} The workaround is to switch the top-level Originally this PR proposed to create the Docker container action as a YAML file with a single placeholder field, like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @woodruffw I requested that. I don't want to have another file with hard-coded contents in the repo. There was a template-like thing before. The published container version is being generated dynamically. |
||||||||||||||
env: | ||||||||||||||
EVENT: ${{ github.event_name }} | ||||||||||||||
REF: ${{ steps.set-repo-and-ref.outputs.ref }} | ||||||||||||||
REPO: ${{ steps.set-repo-and-ref.outputs.repo }} | ||||||||||||||
REPO_ID: ${{ steps.set-repo-and-ref.outputs.repo-id }} | ||||||||||||||
shell: bash | ||||||||||||||
working-directory: action-repo | ||||||||||||||
- name: Run Docker container | ||||||||||||||
uses: ./action-repo/.github/actions/run-docker-container | ||||||||||||||
with: | ||||||||||||||
user: ${{ inputs.user }} | ||||||||||||||
password: ${{ inputs.password }} | ||||||||||||||
repository-url: ${{ inputs.repository-url || inputs.repository_url }} | ||||||||||||||
packages-dir: ${{ inputs.packages-dir || inputs.packages_dir }} | ||||||||||||||
webknjaz marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
verify-metadata: ${{ inputs.verify-metadata || inputs.verify_metadata }} | ||||||||||||||
skip-existing: ${{ inputs.skip-existing || inputs.skip_existing }} | ||||||||||||||
verbose: ${{ inputs.verbose }} | ||||||||||||||
print-hash: ${{ inputs.print-hash || inputs.print_hash }} | ||||||||||||||
attestations: ${{ inputs.attestations }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we incorporate the prior art from the discussions in #45 and accept the 3-segment version as an input, then deduce everything else from that + push the Git tag and advance the proper branches? Perhaps, this could also happen before this PR so it exists as a separate base.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand what you're asking here. You want separate inputs for major, minor, and patch versions? What if you're building the Docker image from the
release/v1
branch? What is the "3-segment version" in that case?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@br3ndonland I was talking about the workflow inputs. Everything else should be computed from them.
The current workflow for me is as follows:
v1.10.1
on top ofunstable/v1
release/v1
tov1.10.1
release/v1.10
tov1.10.1
(or create it)git push --atomic
everything in one goWith this, my 3-segment “input” is
1.10.1
andv1
+v1.10
are being extracted from that. Do you think we could bake containers into this flow with not many changes? This will involve tagging the same image with multiple tags, as I understand. And the end-users referencing versions by both tags and branches should be able to retrieve whatever container there is, right?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@webknjaz thanks for explaining that. I think we're actually pretty close to the desired workflow. There's just a small update needed to the Docker build workflow so it pushes tags with the major and minor version numbers. I've pushed that change.
Your updated workflow would look like this:
v1.10.1
on top ofunstable/v1
release/v1
tov1.10.1
release/v1.10
tov1.10.1
(or create it)git push --atomic
everything in one goghcr.io/pypa/gh-action-pypi-publish:v1.10.1
ghcr.io/pypa/gh-action-pypi-publish:v1.10
ghcr.io/pypa/gh-action-pypi-publish:v1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@br3ndonland so with #45, I wanted to move all these steps inside the automation. For publishing Python projects, I usually use a workflow with
workflow_dispatch
that allows me to type in the desired version to release and that workflow creates the git tag, the gh release, signs stuff and so on.The idea is that tag+branches+gh-release represent the result of release automation being successful and aren't triggers or manual actions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So... you want the Docker build workflow to run automatically when a tag is created? Or when a GitHub Release is created?
Please be more specific here, and please limit the scope of your requests. The goal of this PR is "Build Docker image and push to GHCR." The goal is not to develop your entire release workflow.
PR #45 is currently a draft and I don't think I should have to make this PR dependent on some other draft PR.