Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add semver tagging semantics #214

Merged
merged 36 commits into from
Nov 16, 2020
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
360c428
woops, remove 'break' which made setup-images noop
vito Jun 21, 2020
3375643
check: introduce semver based version scheme
vito Jun 21, 2020
85dd02c
check: split table tests out, move types from main
vito Jul 9, 2020
1f7abb0
check: initial support for checking 'variants'
vito Jul 9, 2020
d46257f
check: make pre-releases opt-in
vito Jul 9, 2020
0787d06
check: respect 'from' version
vito Jul 9, 2020
8475516
check: slight refactor - dedupe registry mirror
vito Jul 9, 2020
47481a3
check: backfill tests for mirror + semver tags
vito Jul 9, 2020
5246138
check: refactor: branch in checkWithRetry
vito Jul 9, 2020
ea197da
refactor: move request/response types to top pkg
vito Jul 9, 2020
fa606e9
out: pushing version tags (optionally w/ variant)
vito Jul 9, 2020
6902917
out: add helpful error if no tag specified
vito Jul 10, 2020
b9e2f44
out: validate version value, sanitize away 'v'
vito Jul 10, 2020
a2e71e7
update README with new semver behavior
vito Jul 14, 2020
6651dd3
check: don't fetch digests for old versions
vito Jul 16, 2020
611afb7
check: re-use transport to prevent repeated auth
vito Jul 17, 2020
c63d1f0
out: fix failing 429 tests
vito Jul 17, 2020
c00b7af
check: only auth once for tag flow, too
vito Jul 22, 2020
5857d92
check: remove unused 'versions' var
vito Jul 24, 2020
f84d798
check: pull plumbing out into Source methods
vito Jul 24, 2020
b5d74b2
in tests: ensure each version has a tag field
vito Jul 24, 2020
1564f5c
in: backfill test for image tagging w/ mirror
vito Jul 24, 2020
8d2d59a
in: save tag/digest no matter what, refactor
vito Jul 24, 2020
b4028e1
Merge branch 'master' into semver-tags
vito Jul 31, 2020
77c9dea
structure: tag vs. ref naming consistency
vito Jul 31, 2020
2d4e623
structure: remove redundant WeakValidation args
vito Jul 31, 2020
ed69157
check: use HEAD requests to avoid rate limits
vito Sep 16, 2020
861e70e
Merge branch 'master' into semver-tags
vito Sep 16, 2020
cb25c2c
Merge branch 'master' into semver-tags
vito Nov 2, 2020
ed142fa
Merge branch 'master' into semver-tags
vito Nov 9, 2020
dd0d1a5
allow 1.2.3-rc1 and 1.2.3-rc as prereleases
vito Nov 9, 2020
db34ba0
final version tag takes priority over prereleases
vito Nov 9, 2020
e11ca27
revise README for prerelease prefix change
vito Nov 10, 2020
5071a4e
fix order dependence for final version precedence
vito Nov 10, 2020
2e1827d
include tag in response in tag checking mode
vito Nov 11, 2020
5f9d3a2
attempt to clarify `variant` + `tag` usage
vito Nov 11, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 134 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
# Registry Image Resource

Supports checking, fetching, and pushing of images to Docker registries.
This resource can be used in two ways: [with `tag`
specified](#check-with-tag-discover-new-digests-for-the-tag) and [without
`tag`](#check-without-tag-discover-semver-tags).

With `tag` specified, `check` will detect changes to the digest the tag points
to, and `out` will always push to the specified tag. This is to be used in
simpler cases where no real versioning exists.

With `tag` omitted, `check` will instead detect tags based on semver versions
vito marked this conversation as resolved.
Show resolved Hide resolved
(e.g. `1.2.3`) and return them in semver order. With `variant` included,
`check` will only detect semver tags that include the variant suffix (e.g.
`1.2.3-stretch`).

## Comparison to `docker-image` resource

This resource is intended as a replacement for the [Docker Image
resource](https://github.com/concourse/docker-image-resource). Here are the key
Expand All @@ -26,12 +40,18 @@ differences:

## Source Configuration

* `repository`: *Required.* The name of the repository, e.g. `alpine`. If using ecr
you only need the repository name, not the full URI e.g. `alpine` not
`012345678910.dkr.ecr.us-east-1.amazonaws.com/alpine`
* `repository`: *Required.* The name of the repository, e.g. `alpine` or
`concourse/concourse`.

*Note: If using ecr you only need the repository name, not the full URI e.g.
`alpine` not `012345678910.dkr.ecr.us-east-1.amazonaws.com/alpine`*

* `tag`: *Optional.* Instead of monitoring semver tags, monitor a single tag
for changes (based on digest).

* `tag`: *Optional. Default `latest`.* The name of the tag to monitor and
publish to.
* `variant`: *Optional.* Detect only tags matching this variant suffix, and
aoldershaw marked this conversation as resolved.
Show resolved Hide resolved
push version tags with this suffix applied. For example, a value of
`stretch` would be used for tags like `1.2.3-stretch`.

* `username` and `password`: *Optional.* A username and password to use when
authenticating to the registry. Must be specified for private repos or when
Expand Down Expand Up @@ -109,15 +129,108 @@ registry_key: |

## Behavior

### `check`: Discover new digests.
### `check` with `tag`: discover new digests for the tag

Reports the current digest that the registry has for the tag configured in
`source`.

### `check` without `tag`: discover semver tags

Detects tags which contain semver version numbers. Version numbers do not
need to contain all 3 segments (major/minor/patch).

Each unique digest will be returned only once, with the most specific version
tag available. This is to handle "alias" tags like `1`, `1.2` pointing to
`1.2.3`.

Note: the initial `check` call will return *all valid versions*, which is
unlike most resources which only return the latest version. This is an
intentional choice which will become the normal behavior for resources in
the future (per concourse/rfcs#38).

Example:

```yaml
resources:
- name: concourse
type: registry-image
source: {repository: concourse/concourse}
```

The above resource definition would detect the following versions:

```json
[
{
"tag": "1.6.0",
"digest": "sha256:e1ad01d3227569ad869bdb6bd68cf1ea54057566c25bae38b99d92bbe9f28d78"
},
{
"tag": "2.0.0",
"digest": "sha256:9ab8d1021d97c6602abbb2c40548eab67aa7babca22f6fe33ab80f4cbf8ea92c"
},
// ...
]
```

#### Variant tags

Docker repositories have a pretty common convention of adding `-SUFFIX` to
tags to denote "variant" images, i.e. the same version but with a different
base image or dependency. For example, `1.2.3` vs `1.2.3-alpine`.

### `in`: Fetch the image's rootfs and metadata.
With a `variant` value specified, only semver tags with the matching variant
will be detected. With `variant` omitted, tags which include a variant are
ignored.

Fetches an image at a digest.
Note: some image tags actually include *mutliple* variants, e.g.
`1.2.3-php7.3-apache`. With a variant of only `apache` configured, these tags
will be skipped to avoid accidentally using multiple variants. In order to
use these tags, you must specify the full variant combination, e.g.
`php7.3-apache`.

Example:

```yaml
resources:
- name: concourse
type: registry-image
source:
repository: concourse/concourse
variant: ubuntu
```

The above resource definition would detect the following versions:

```json
[
{
"tag": "5.2.1-ubuntu",
"digest": "sha256:91f5d180d84ee4b2cedfae45771adac62c67c3f5f615448d3c34057c09404f27"
},
{
"tag": "5.2.2-ubuntu",
"digest": "sha256:cb631d788797f0fbbe72a00afd18e5e4bced356e1b988d1862dc9565130a6226"
},
// ...
]
```

#### Pre-release versions

By default, pre-release versions are ignored. With `pre_releases: true`, they
will be included.

Note however that variants and pre-releases both use the same syntax:
`1.2.3-alpine` is technically also valid syntax for a Semver prerelease. For
this reason, the resource will only consider prerelease data starting with
`alpha`, `beta`, or `rc` as a proper prerelease, treating anything else as
a variant.


### `in`: fetch an image

Fetches an image at the exact digest specified by the version.

#### Parameters

Expand All @@ -129,8 +242,8 @@ Fetches an image at a digest.

The resource will produce the following files:

* `./digest`: A file containing the image's digest, e.g. `sha256:...`.
* `./tag`: A file containing the tag from `source`, e.g. `latest`.
* `./tag`: A file containing the tag from the version.
* `./digest`: A file containing the digest from the version, e.g. `sha256:...`.

The remaining files depend on the configuration value for `format`:

Expand All @@ -156,20 +269,26 @@ In this format, the resource will produce the following files:
* `./image.tar`: the OCI image tarball, suitable for passing to `docker load`.


### `out`: Push an image up to the registry under the given tags.

Uploads an image to the registry under the tag configured in `source`.
### `out`: push and tag an image

If `additional_tags` param is defined then the uploaded image will also be
tagged with each one of the values specified in that file.
Pushes an image to the registry as the specified tags.

The currently encouraged way to build these images is by using the
[`oci-build-task`](https://github.com/vito/oci-build-task).

Tags may be specified in multiple ways:

* With `tag` configured in `source`, the configured tag will always be pushed.
* With `version` given in `params`, the image will be pushed using the version
number as a tag, optionally with a `variant` suffix (configured in `source`).
* With `additional_tags` given in `params`, the image will be pushed as each
tag listed in the file (whitespace separated).

#### Parameters

* `image`: *Required.* The path to the OCI image tarball to upload. Expanded
with [`filepath.Glob`](https://golang.org/pkg/path/filepath/#Glob).
* `version`: *Optional.* A version number to use as a tag.
* `additional_tags`: *Optional.* The path to a file with whitespace-separated
list of tag values to tag the image with (in addition to the tag configured
in `source`).
Expand Down
Loading