Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi arch docker build #104

Merged
merged 51 commits into from
Aug 3, 2024
Merged
Changes from 16 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
b04422c
Add GitHub Actions workflow for multi-arch Docker build
sharonsyh Jul 25, 2024
8bacff9
Edited build action version
sharonsyh Jul 26, 2024
1626d4b
Add GitHub Actions workflow for multi-arch Docker build
sharonsyh Jul 26, 2024
8c343db
Modified tag formats
sharonsyh Jul 26, 2024
fef9050
Removed unneccessary file build.yml -> push_docker.yaml
sharonsyh Jul 26, 2024
085b527
Removed docker login part
sharonsyh Jul 26, 2024
6adaad9
Fixed syntax error
sharonsyh Jul 26, 2024
4a9b348
Added docker login
sharonsyh Jul 26, 2024
e73c8a7
Fixed syntax error
sharonsyh Jul 26, 2024
6f70a0e
Added master branch for the workflow
sharonsyh Jul 26, 2024
2244b55
Build Trial
sharonsyh Jul 26, 2024
5a6dc5e
Build Trial
sharonsyh Jul 26, 2024
412b8ed
Version control
sharonsyh Jul 26, 2024
9b12b24
Version control
sharonsyh Jul 26, 2024
031af2e
Workflow order modified
sharonsyh Jul 26, 2024
bfed73f
Removed build-args
sharonsyh Jul 26, 2024
352b133
Changed the branch to the master
sharonsyh Jul 30, 2024
8edef7f
Modified build-push-action to directly specify the platform
sharonsyh Jul 31, 2024
860ef41
Added branch multi-arch-docker-build to push section in order to test…
sharonsyh Jul 31, 2024
f254c13
Syntax error fixed
sharonsyh Jul 31, 2024
bee2927
Workflow modified to enavle distribute build across multiple runners
sharonsyh Jul 31, 2024
7b33c07
Removed push-by-digest
sharonsyh Jul 31, 2024
7b38f14
Downgraded the version for upload-artifact in order to upload to same…
sharonsyh Jul 31, 2024
608a31a
Modified the username for docker login
sharonsyh Jul 31, 2024
2c95e8b
Syntax error fixed
sharonsyh Jul 31, 2024
bb5c4a6
Added a step to create /tmp/digests directory before using it in both…
sharonsyh Jul 31, 2024
ec8c103
Syntax error fixed
sharonsyh Jul 31, 2024
57865c2
Added outputs parameter for capturing the digest
sharonsyh Aug 1, 2024
1314ea6
Syntax error fixed
sharonsyh Aug 1, 2024
69a0fc6
Removed manual tags option
sharonsyh Aug 1, 2024
df86aa8
Defined environment variable PLATFORM_PAIR
sharonsyh Aug 1, 2024
db2f55c
Seperated commands for creating manifest list
sharonsyh Aug 1, 2024
99ffbd7
Added - name: Prepare
sharonsyh Aug 1, 2024
b014f4f
Additional directory creation deleted
sharonsyh Aug 1, 2024
1058e7e
Modified export command from touch to echo
sharonsyh Aug 1, 2024
f43b19d
Shorten the code for creating manifest list
sharonsyh Aug 1, 2024
9f6540b
Syntax error fixed
sharonsyh Aug 1, 2024
8de8170
Added a line to look for the files /tmp/digests/*.digest
sharonsyh Aug 1, 2024
bc19e06
eval command added for execution of the command
sharonsyh Aug 1, 2024
ef61bd4
Specified digests' name directly when downloading
sharonsyh Aug 1, 2024
6c4278a
Downgraded download-artifact to ensure compatibility with upload-arti…
sharonsyh Aug 1, 2024
cb3247e
Fixed bad substitution error
sharonsyh Aug 1, 2024
0ff2f69
Fixed how digest files are accessed
sharonsyh Aug 1, 2024
156cc86
Modified in order to download each artifact individually
sharonsyh Aug 1, 2024
89004e7
Modified in order to download each artifact individually
sharonsyh Aug 1, 2024
1de5c50
Modified to avoid SHA256 prefix duplication
sharonsyh Aug 1, 2024
d47970a
Upgraded the version for login-action
sharonsyh Aug 1, 2024
fca2fa2
modified the command from touch to echo
sharonsyh Aug 1, 2024
e7b0aea
Upgraded upload-digest and download-digest version
sharonsyh Aug 3, 2024
0337932
Removed multi-arch-docker-build branch from on-push branch
sharonsyh Aug 3, 2024
9230a86
Removed fail-fast from the strategy line
sharonsyh Aug 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 23 additions & 15 deletions .github/workflows/push_docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Push Docker image
on:
push:
branches:
- master
- multi-arch-docker-build
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be reverted back to master now, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hasn't been addressed. Please do a thorough scan of the workflow file to ensure that it can land in master and request review.

tags:
- zeus-v*
paths:
Expand All @@ -18,11 +18,20 @@ on:
- 'LICENSE'
- 'setup.py'
- 'pyproject.toml'

pull_request:
branches:
- multi-arch-docker-build
jaywonchung marked this conversation as resolved.
Show resolved Hide resolved
jobs:
build_and_push:
if: github.repository_owner == 'ml-energy'
runs-on: ubuntu-latest
strategy:
fail-fast: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It ensures that both platform builds will run to completion, regardless of whether one of them fails. Do you think this is unnecessary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is fail-fast true by default? I think we actually want to fail fast. If one platform failed, it means something is wrong with the Dockerfile and we don't want to upload a partial image.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have set fail-fast value to true as it is not true by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, it seems to be true by default. Removed the line!

matrix:
platform:
- linux/amd64
- linux/arm64

steps:
- name: Remove unnecessary files
run: |
Expand All @@ -31,33 +40,32 @@ jobs:
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
- name: Checkout repository
uses: actions/checkout@v3
- name: Docker Hub login
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The master branch's workflow file would need this, one way or another. In general it seems like this PR is not ready to be merged to master. Please update the workflow file so that it is as it should be on the master branch after merging.

uses: actions/checkout@v4
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Generate image metadata
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
symbioticlab/zeus
mlenergy/zeus
tags: |
type=ref,event=branch
type=ref,event=tag
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v2
type=schedule
type=sha
jaywonchung marked this conversation as resolved.
Show resolved Hide resolved
- name: Build and push to Docker Hub
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/Dockerfile
builder: ${{ steps.buildx.outputs.name }}
push: true
push: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=mlenergy/zeus:buildcache
cache-to: type=registry,ref=mlenergy/zeus:buildcache,mode=max
build-args: TARGETARCH=amd64
platforms: ${{ matrix.platform }}

Loading