Stamp for docker_push?

[samuraisam]

In my docker/BUILD file I have something like this to create docker images. Currently this requires bazel clean between builds for these services because I haven't figured out how to get the stamps to update every run (which is besides the point, there must be a way).

package(default_visibility = ["//visibility:public"])

load("@io_bazel_rules_docker//docker:docker.bzl", "docker_bundle", "docker_push")

DOCKER_BUILDS = {
    "assets":       "//assets/assets:docker", # etc...
}

[docker_bundle(
    name = binary,
    images = {"gcr.io/{GCLOUD_PROJECT_ID}/%s:{BRANCH_NAME}-{BUILD_NUMBER}" % binary: docker_image},
    stamp = True
 ) for binary, docker_image in DOCKER_BUILDS.items()]

My desire is to define matching docker_push targets:

[docker_push(
    name = "push_" + binary,
    image = ":" + binary,
    registry = "gcr.io",
    repository = "{GCLOUD_PROJECT_ID}/%s" % binary,
    tag = "{BRANCH_NAME}-{BUILD_NUMBER}",
    stamp = true,
 ) for binary in DOCKER_BUILDS.keys()]

However this doesn't appear to be supported. I assume there is a reason why that is escaping me. Or maybe I should just be doing this in an external tool.

[mattmoor]

This is the final bit remaining here: #3

I just need to incorporate a trick like what @ixdy did for docker_bundle into the docker_pusher tool.

I'll investigate whether this is as simple as I think today, and post back if there are any unforeseen problems.

[mattmoor]

@samuraisam Hopefully my PR addresses what you're looking for?

[samuraisam]

Based on the test in your PR this looks like it would work for my use case above other than still needing to hardcode the repository (eg {GCLOUD_PROJECT_ID}) but maybe that could be incorporated into the tag?

[mattmoor]

@ixdy is a bit more familiar than I am with manipulating the workspace status stuff, so perhaps he has a suggestion?

[ixdy]

the test in the PR didn't cover it, but I just tested manually where I used {BUILD_USER} in the repository and {STABLE_DOCKER_TAG} (a custom value) in the tag and it worked as expected.

[samuraisam]

Neato, thanks! Is there a way to get this to work without a bazel clean between builds, or is that not the way stamps are supposed to work?

[mattmoor]

No clue. @damienmg ?

[ixdy]

@samuraisam what do you mean?

If I'm reading your comments correctly, it looks like you are saving GCLOUD_PROJECT_ID in your workspace status and then trying to use that to create the tag. Is that correct?

If so, I think you might be running into some (likely undocumented) behaviors of how the workspace status/stamping works in Bazel.

There's some discussion in bazelbuild/bazel#1758, but basically bazel devices the values into two files, stable-status.txt and volatile-status.txt. Stable values like the build label, build host, and build user go into stable-status.txt; volatile values like the build timestamp go into volatile-status.txt.

All of the stamping logic (at least rules_docker and rules_go :) looks at both of these files, but only changes to stable-status.txt will trigger rebuilds/restamps. That way if you run bazel multiple times, it won't force a rebuild because the time changed.

The undocumented subtleties apply with custom workspace values. Only values prefixed with STABLE_ end up in the stable-status.txt. Since your custom value GCLOUD_PROJECT_ID isn't prefixed with STABLE_, it ends up in volatile-status.txt, and changes to it won't trigger rebuilds.

[ixdy]

@samuraisam also, depending how you are defining GCLOUD_PROJECT_ID, it may make more sense to use "make" variables.

For example, you can do something like this:

docker_push(
    name = "push-image",
    image = ":image",
    registry = "gcr.io",
    repository = "$(PROJECT)/my-image",
    tag = "dev",
)

and then set the project at run-time by doing bazel run push-image --define PROJECT=my-project.

(You can also combine "make" variables and stamps. e.g. maybe tag uses the git hash, but the project is specified on the cmdline.)

[mattmoor]

@ixdy I actually think the Bazel folks didn't open source the vardef/varref stuff for some strange reason, so we can only use the make variables internally. Have you seen otherwise?

[ixdy]

I tested the example I pasted and it worked.

I guess vardef/varref don't exist, but --define sets things in in the namespace used by ctx.expand_make_variables?

[ixdy]

... which I think is roughly what you were saying. Someone writing a BUILD file can't arbitrarily expand "make" variables wherever they want, but since the docker_push rule uses ctx.expand_make_variables internally, they're supported there.

[mattmoor]

That's interesting, so I guess you can use it, but not with default values? Weird.

[samuraisam]

Thanks for the advice. I'm learning more about bazel every day! The primary concern here is that we tag builds with the BUILD_NUMBER eg $PROJECT:$BRANCH_NAME-$BUILD_NUMBER and when $BUILD_NUMBER changes we aren't getting a rebuild. From what I understand this is intended, since bazel produces hermetic builds, we can't change arbitrary values that aren't baked into the environment.

To work with this I just run a script afterward that runs docker load on the generated tar files and then docker tag <X> <Y> followed by the docker push command. This works, but is slow and has to run for every one of our docker images (which is currently at 8 but soon to be at 10 and won't always be scaleable to do it this way).

[ixdy]

if you are saving those variables directly into the workspace status, i.e. your workspace_status_command looks something like

#!/bin/sh
echo PROJECT $PROJECT
echo BRANCH_NAME $BRANCH_NAME
echo BUILD_NUMBER $BUILD_NUMBER

try prefixing those with STABLE_, i.e

#!/bin/sh
echo STABLE_PROJECT $PROJECT
echo STABLE_BRANCH_NAME $BRANCH_NAME
echo STABLE_BUILD_NUMBER $BUILD_NUMBER

and then use {STABLE_PROJECT}:{STABLE_BRANCH_NAME}-${STABLE_BUILD_NUMBER} in the docker rule. That should trigger re-stamping as the values change, though most of your build should remain cached run-to-run.

[samuraisam]

Oh that's awesome! I really appreciate the advice. I do have a workspace_status_command but didn't know about STABLE_ (though I somehow avoided noticing it in the kubernetes project)