build(operator-sdk): upgrade operator sdk version to 1.28.0 #543

tthvo · 2023-03-28T21:12:00Z

Fixes #536
Fixes #455

Upgrade to use operator sdk 1.28.0. Step by step changes:

References:

1.23.0

Update controller-gen from 0.9.0 to 0.9.2. Update to use variable CONTROLLER_TOOLS_VERSION for controller-gen version (i.e. sync with operator-sdk go template).
Update controller-runtime from 0.12.1 to 0.12.2 and Kubernetes dependencies from 0.24.0 to 0.24.2.
Upgrade kube proxy image gcr.io/kubebuilder/kube-rbac-proxy from 0.11.0 to 0.13.0.

1.24.0

Nothing to change. There are some sed ops on ARCH var for arm support but no mentions of Go operators.

1.25.0

Bump k8s runtime dependencies to 0.25.0 and controller-runtime to 0.13.0.
- Upgrade go version to 1.19.
- Update test client Get func signature.
Bump test dependencies (mainly to Gingko v2).
- Remove custom report (deprecated) for adding newline.
- Remove ACK_GINKGO_DEPRECATIONS as Gingko is upgrade to v2.
Multi-platform build for operator image. docker only and need to install buildx.

1.26.0

Nothing to change.

1.27.0

Nothing to change.

1.28.0

Makefile update
- Bump CONTROLLER_TOOLS_VERSION to 0.11.1 (migration guide said 0.11.3 but scaffold specifies at 0.11.1).
- Bump ENVTEST_K8S_VERSION to 1.26.
- Update prerequisites of manager target to include manifests target. Update image build targets to inlude fmt and vet.
- controller-gen and kustomize is added with a check for version (i.e. download if version is mismatched).
Bump k8s runtime dependencies to 0.26.2 and controller-runtime to 0.14.5.
Bump kube-rbac-proxy image to 0.13.1.
Set --pod-security=restricted when running operator-sdk scorecard. This launches scorecard pods with restricted PSA.

Tests

Reference bundle image: quay.io/thvo/cryostat-operator-bundle:latest.
Passed scorecards.

tthvo · 2023-04-11T20:09:11Z

For multi-arch build, it needs official docker tool with buildx plugin to build. Took me a while to realize the requirement for official docker includes Fedora 36, but I am still on Fedora 35 and using moby-engine. Not sure there are enough time to upgrade to test. Could you help check it out instead?

Also, wondering if we need to add multi-arch build for scorecard images?

Makefile

ebaron · 2023-04-12T15:25:28Z

For multi-arch build, it needs official docker tool with buildx plugin to build. Took me a while to realize the requirement for official docker includes Fedora 36, but I am still on Fedora 35 and using moby-engine. Not sure there are enough time to upgrade to test. Could you help check it out instead?

Also, wondering if we need to add multi-arch build for scorecard images?

I was using moby-engine with the buildx plugin installed into ~/.docker/cli-plugins/: https://github.com/docker/buildx#manual-download

We will eventually need multi-arch scorecard images too, but it doesn't need to be done with this PR.

tthvo · 2023-04-12T16:49:07Z

I was using moby-engine with the buildx plugin installed into ~/.docker/cli-plugins/: https://github.com/docker/buildx#manual-download

Oh thanks! I will try that (saw the word not recommend last time so I quickly skipped it :D ).

EDIT: Nice got it to use buildx with moby engine now. Thanks alot!

tthvo · 2023-04-12T17:13:12Z

We also need to specify supported OS and ARCH in CSV? https://olm.operatorframework.io/docs/advanced-tasks/ship-operator-supporting-multiarch/

README.md

andrewazores · 2023-04-12T17:57:14Z

EDIT: Nice got it to use buildx with moby engine now. Thanks alot!

Any way around the create/use temporary build contexts? If so then this would also be podman compatible, which would be nice...

ebaron · 2023-04-12T17:57:14Z

We also need to specify supported OS and ARCH in CSV? https://olm.operatorframework.io/docs/advanced-tasks/ship-operator-supporting-multiarch/

We do yes, but this may be a bit premature until the operator and all the operand images have published multi-arch images.

tthvo · 2023-04-12T18:00:38Z

Any way around the create/use temporary build contexts? If so then this would also be podman compatible, which would be nice...

I am reading this: https://github.com/docker/buildx#working-with-builder-instances. Without creating new builder, there is an error:

ERROR: multiple platforms feature is currently not supported for docker driver. 
Please switch to a different driver (eg. "docker buildx create --use")
make: *** [Makefile:245: oci-buildx] Error 1

Also, seems like docker-buildx needs --push option to be specified. Podman does not have it too?

Also, for Buildah, we need to use manifest list

--platform="OS/ARCH[/VARIANT]"
The --platform flag can be specified more than once, or given a comma-separated list of values as its argument.
When more than one platform is specified, the --manifest option should be used instead of the --tag option.

How about split cases for docker and podman so we can specified command separately?

ebaron · 2023-04-12T18:12:04Z

How about split cases for docker and podman so we can specified command separately?

That seems fine to me, just wrap the extra Docker stuff in ifeq($(IMAGE_BUILDER),docker)

Dockerfile

tthvo · 2023-04-12T19:14:15Z

I am having a bit of issue with oc-build, where the command

[1/2] STEP 11/11: RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} GO111MODULE=on go build -a -o manager internal/main.go

takes forever to run. Tho, it eventually finishes (happens on first build after bumping go and deps). Wondering if its just my setup or the same for everyone? Or was it because builder container was cached?

ebaron · 2023-04-12T20:24:23Z

I am having a bit of issue with oc-build, where the command
[1/2] STEP 11/11: RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} GO111MODULE=on go build -a -o manager internal/main.go
takes forever to run. Tho, it eventually finishes (happens on first build after bumping go and deps). Wondering if its just my setup or the same for everyone? Or was it because builder container was cached?

It doesn't take long for me. Not sure why there's a difference.

Edit: Just noticed you mentioned after bumping the deps once. That's because the layer that downloads all the new dependencies is cached. So subsequent builds will be faster.

andrewazores · 2023-04-12T20:25:02Z

I find that takes quite a while too, and subsequent runs are a lot faster after that container layer is cached. podman image prune -f deletes it because the intermediate is not tagged, and then the next invocation takes a long time again.

That's the case on main though, not just for this PR (which I haven't tested yet)

tthvo · 2023-04-12T20:33:40Z

Oh yeh exactly! I have been running prune after every build. That makes sense thanks a lot!

tthvo · 2023-04-12T20:41:28Z

Added the handing for podman case. Seems to build as expected (only included 2 amd64 and arm64. ppc64le and s390x takes way to long):

https://quay.io/repository/thvo/cryostat-operator/manifest/sha256:bdda2f96fd1d1c885616ea8eef18a724a8dcf1e73ef8ff5ac799231434d9dc65

For podman, I used the original Dockerfile as it seems to build using emulator, not cross-platform compilation.

Makefile

ebaron

Looks good to me! Thanks @tthvo!

tthvo added the build label Mar 28, 2023

tthvo force-pushed the operator-sdk branch 8 times, most recently from 2905232 to d6ae766 Compare April 11, 2023 19:50

tthvo marked this pull request as ready for review April 11, 2023 20:07

tthvo requested review from andrewazores and ebaron April 11, 2023 20:07

tthvo commented Apr 11, 2023

View reviewed changes

Makefile Show resolved Hide resolved

tthvo commented Apr 11, 2023

View reviewed changes

Makefile Show resolved Hide resolved

andrewazores reviewed Apr 11, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

ebaron reviewed Apr 11, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Thuan Vo added 6 commits April 11, 2023 18:45

build(operator-sdk): upgrade operator sdk version to 1.23.0

1760629

build(operator-sdk): upgrade operator sdk version to 1.24.0

5bad0cd

build(operator-sdk): upgrade operator sdk version to 1.25.0

04243ef

build(operator-sdk): upgrade operator sdk version to 1.26.0

19fad4e

build(operator-sdk): upgrade operator sdk version to 1.27.0

de1d713

build(operator-sdk): upgrade operator sdk version to 1.28.0

452d1c6

chore(makefile): use PHONEY for local bin

e94a854

tthvo force-pushed the operator-sdk branch from d6ae766 to e94a854 Compare April 12, 2023 17:00

ebaron reviewed Apr 12, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

chore(docs): update README

24b5197

tthvo commented Apr 12, 2023

View reviewed changes

Dockerfile Show resolved Hide resolved

fix(makefile): split multi-arch between docker and podman

5ed82b4

tthvo force-pushed the operator-sdk branch from f734e72 to 5ed82b4 Compare April 12, 2023 20:40

ebaron reviewed Apr 12, 2023

View reviewed changes

Makefile Show resolved Hide resolved

ebaron reviewed Apr 12, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

fix(makefile): uncomment prereq and check podman

fcdaf6f

tthvo force-pushed the operator-sdk branch from bb68c06 to fcdaf6f Compare April 13, 2023 18:07

ebaron approved these changes Apr 13, 2023

View reviewed changes

ebaron added the chore Refactor, rename, cleanup, etc. label Apr 13, 2023

ebaron merged commit cd14562 into cryostatio:main Apr 13, 2023

tthvo deleted the operator-sdk branch April 13, 2023 18:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build(operator-sdk): upgrade operator sdk version to 1.28.0 #543

build(operator-sdk): upgrade operator sdk version to 1.28.0 #543

tthvo commented Mar 28, 2023 •

edited

Loading

tthvo commented Apr 11, 2023 •

edited

Loading

ebaron commented Apr 12, 2023 •

edited

Loading

tthvo commented Apr 12, 2023 •

edited

Loading

tthvo commented Apr 12, 2023

andrewazores commented Apr 12, 2023

ebaron commented Apr 12, 2023

tthvo commented Apr 12, 2023 •

edited

Loading

ebaron commented Apr 12, 2023

tthvo commented Apr 12, 2023 •

edited

Loading

ebaron commented Apr 12, 2023 •

edited

Loading

andrewazores commented Apr 12, 2023

tthvo commented Apr 12, 2023

tthvo commented Apr 12, 2023 •

edited

Loading

ebaron left a comment

build(operator-sdk): upgrade operator sdk version to 1.28.0 #543

build(operator-sdk): upgrade operator sdk version to 1.28.0 #543

Conversation

tthvo commented Mar 28, 2023 • edited Loading

1.23.0

1.24.0

1.25.0

1.26.0

1.27.0

1.28.0

Tests

tthvo commented Apr 11, 2023 • edited Loading

ebaron commented Apr 12, 2023 • edited Loading

tthvo commented Apr 12, 2023 • edited Loading

tthvo commented Apr 12, 2023

andrewazores commented Apr 12, 2023

ebaron commented Apr 12, 2023

tthvo commented Apr 12, 2023 • edited Loading

ebaron commented Apr 12, 2023

tthvo commented Apr 12, 2023 • edited Loading

ebaron commented Apr 12, 2023 • edited Loading

andrewazores commented Apr 12, 2023

tthvo commented Apr 12, 2023

tthvo commented Apr 12, 2023 • edited Loading

ebaron left a comment

Choose a reason for hiding this comment

tthvo commented Mar 28, 2023 •

edited

Loading

tthvo commented Apr 11, 2023 •

edited

Loading

ebaron commented Apr 12, 2023 •

edited

Loading

tthvo commented Apr 12, 2023 •

edited

Loading

tthvo commented Apr 12, 2023 •

edited

Loading

tthvo commented Apr 12, 2023 •

edited

Loading

ebaron commented Apr 12, 2023 •

edited

Loading

tthvo commented Apr 12, 2023 •

edited

Loading