Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi arch docker build #104

Merged
merged 51 commits into from
Aug 3, 2024
Merged

Multi arch docker build #104

merged 51 commits into from
Aug 3, 2024

Conversation

sharonsyh
Copy link
Collaborator

Summary
This pull request introduces a GitHub Actions workflow to build and push multi-platform Docker images for the Zeus project. The workflow supports both linux/amd64 and linux/arm64 architectures.

Testing and Future Integration
This workflow is currently set up to test on the multi-arch-docker-build branch. The 401 error encountered during the build process is expected to be resolved by including Docker login credentials and pushing to the master branch. Once verified, this workflow will be integrated into the master branch to enable seamless multi-platform image builds and pushes.

Copy link
Member

@jaywonchung jaywonchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I looked at the GitHub Action logs for the pull request and things look like they're working.

Could you now update the workflow file so that it can be merged to master?
For instance, Docker credential secrets will have to be brought back into the workflow file. I've marked other places that need modification so that it can land on master.

@@ -3,7 +3,7 @@ name: Push Docker image
on:
push:
branches:
- master
- multi-arch-docker-build
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be reverted back to master now, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hasn't been addressed. Please do a thorough scan of the workflow file to ensure that it can land in master and request review.

.github/workflows/push_docker.yaml Outdated Show resolved Hide resolved
jobs:
build_and_push:
if: github.repository_owner == 'ml-energy'
runs-on: ubuntu-latest
strategy:
fail-fast: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It ensures that both platform builds will run to completion, regardless of whether one of them fails. Do you think this is unnecessary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is fail-fast true by default? I think we actually want to fail fast. If one platform failed, it means something is wrong with the Dockerfile and we don't want to upload a partial image.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have set fail-fast value to true as it is not true by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, it seems to be true by default. Removed the line!

.github/workflows/push_docker.yaml Outdated Show resolved Hide resolved
Comment on lines 36 to 39
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The master branch's workflow file would need this, one way or another. In general it seems like this PR is not ready to be merged to master. Please update the workflow file so that it is as it should be on the master branch after merging.

@jaywonchung jaywonchung linked an issue Jul 29, 2024 that may be closed by this pull request
@jaywonchung
Copy link
Member

jaywonchung commented Jul 30, 2024

https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners
Shouldn't we have a merge job like the example in the link above?

@jaywonchung
Copy link
Member

Screenshot 2024-07-30 at 9 57 12 AM

What we want to have is images like this: One tag supports two different architectures. Unless I'm mistaken, I think the GitHub Actions workflow we currently have will create an ARM64 and an AMD64 image separately (independent image with just a single architecture) with the same tag, and the one that finished later will overwrite the one that finished earlier, and we'll end up with either a single-arch ARM64 image or a single-arch AMD64 image.

So apparently there are two ways to do this:

Please try both and see which one finishes earlier. We'll use the one that finishes quicket.

Copy link
Member

@jaywonchung jaywonchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for your work! Let's see if things look fine on the master branch.

@jaywonchung jaywonchung merged commit 7d31794 into master Aug 3, 2024
@jaywonchung jaywonchung deleted the multi-arch-docker-build branch August 3, 2024 03:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatic multi-arch Docker images in CI
2 participants