Skip to content

Commit

Permalink
Implement publish workflow (#5)
Browse files Browse the repository at this point in the history
<!-- markdownlint-disable-next-line first-line-heading -->
## Description

<!-- Describe your changes in detail. -->

## Context

<!-- Why is this change required? What problem does it solve? -->

## Type of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply. -->

- [ ] Refactoring (non-breaking change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would change existing
functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->

- [ ] I am familiar with the [contributing
guidelines](../docs/CONTRIBUTING.md)
- [ ] I have followed the code style of the project
- [ ] I have added tests to cover my changes
- [ ] I have updated the documentation accordingly
- [ ] This PR is a result of pair or mob programming

---

## Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others
privacy, we kindly ask you to NOT including [PII (Personal Identifiable
Information) / PID (Personal Identifiable
Data)](https://digital.nhs.uk/data-and-information/keeping-data-safe-and-benefitting-the-public)
or any other sensitive data in this PR (Pull Request) and the codebase
changes. We will remove any PR that do contain any sensitive
information. We really appreciate your cooperation in this matter.

- [ ] I confirm that neither PII/PID nor sensitive data are included in
this PR and the codebase changes.
  • Loading branch information
stefaniuk committed Sep 9, 2023
1 parent 8a0f5a8 commit 2e26ba9
Show file tree
Hide file tree
Showing 11 changed files with 92 additions and 33 deletions.
20 changes: 18 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,28 @@ include ./scripts/test.mk
DOCKER_IMAGE := ghcr.io/nhs-england-tools/github-runner-image
DOCKER_TITLE := "GitHub Runner Image"

build: # Build Docker image - optional: name=[image name to build, default is 'base']
# ==============================================================================

build: # Build Docker images
make build-image name="base"
make build-image name="rt"

publish: # Publish Docker images
make publish-image name="base"
make publish-image name="rt"

# ==============================================================================

build-image: # Build Docker image - optional: name=[image name to build, default is 'base']
dir=infrastructure/images/$(or ${name}, "base") make docker-build

publish: # Publish Docker image - optional: name=[image name to publish, default is 'base']
publish-image: # Publish Docker image - optional: name=[image name to publish, default is 'base']
dir=infrastructure/images/$(or ${name}, "base") make docker-push

# ==============================================================================

.SILENT: \
build \
build-image \
publish \
publish-image \
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GitHub Runner Image
# GitHub Runner Image (PoC / Experimental)

[![Publish](https://github.com/nhs-england-tools/github-runner-image/actions/workflows/cicd-2-publish.yaml/badge.svg)](https://github.com/nhs-england-tools/github-runner-image/actions/workflows/cicd-2-publish.yaml)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=github-runner-image&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=github-runner-image)
Expand All @@ -7,7 +7,7 @@ TODO: Overview

## Table of Contents

- [GitHub Runner Image](#github-runner-image)
- [GitHub Runner Image (PoC / Experimental)](#github-runner-image-poc--experimental)
- [Table of Contents](#table-of-contents)
- [Setup](#setup)
- [Prerequisites](#prerequisites)
Expand Down
8 changes: 8 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# TODO

- Check the [actions-runner](https://github.com/actions/runner/pkgs/container/actions-runner) image
- Check the [actions-runner-controller](https://github.com/actions/actions-runner-controller)
- Finish off publishing the Docker image using an automated process
- Create `docker-push` make target
- move docker shell functions to RT
- rename `_get-version` to `_get-effective-version`
- rename `_get-all-versions` to `_get-all-effective-versions`
- `version-create-effective-file`
- `_replace-image-latest-by-specific-version`
- Lint the Dockerfile
- Spec test the Docker image
- Produce SBOM and scan for CVEs the Docker image
Expand All @@ -10,4 +17,5 @@
- Sonar badge should come from main branch
- NHSE Update from Template
- This CI/CD pipeline has some changes to the RT skeleton, consider porting them back; e.g. a notification should be sent when an artefact is published, not built
- Align naming of jobs and steps
- Write an ADR on the Linux distro choice; i.e. compare Ubuntu, Debian and Alpine
1 change: 0 additions & 1 deletion infrastructure/images/act/Dockerfile

This file was deleted.

3 changes: 0 additions & 3 deletions infrastructure/images/act/VERSION

This file was deleted.

6 changes: 5 additions & 1 deletion infrastructure/images/base/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM ubuntu:22.04@sha256:56887c5194fddd8db7e36ced1c16b3569d89f74c801dc8a5adbf48236fb34564
FROM --platform=linux/amd64 ubuntu:22.04@sha256:56887c5194fddd8db7e36ced1c16b3569d89f74c801dc8a5adbf48236fb34564

ENV \
# SEE: https://docs.docker.com/release-notes/
Expand Down Expand Up @@ -61,3 +61,7 @@ RUN set -eux && \
docker-compose --version && \
docker buildx version && \
node --version

RUN set -eux && \
\
git config --global --add safe.directory '*'
4 changes: 2 additions & 2 deletions infrastructure/images/base/VERSION
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
base
yyyymmdd-base
yyyymmdd-hash-base
${yyyy}${mm}${dd}-base
${yyyy}${mm}${dd}-${hash}-base
File renamed without changes.
10 changes: 10 additions & 0 deletions infrastructure/images/rt/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM --platform=linux/amd64 ghcr.io/nhs-england-tools/github-runner-image:${yyyy}${mm}${dd}-${hash}-base

RUN set -eux && \
\
# Fetch the latest version of the package list
apt update --yes && \
\
# Install development tools
apt install --yes \
gh
3 changes: 3 additions & 0 deletions infrastructure/images/rt/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
rt
${yyyy}${mm}${dd}-rt
${yyyy}${mm}${dd}-${hash}-rt
66 changes: 44 additions & 22 deletions scripts/docker/docker.lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,14 @@ function docker-build() {
--build-arg GIT_BRANCH="$(_get-git-branch-name)" \
--build-arg GIT_COMMIT_HASH="$(git rev-parse --short HEAD)" \
--build-arg BUILD_DATE="$(date -u +"%Y-%m-%dT%H:%M:%S%z")" \
--build-arg BUILD_VERSION="$(_get-version)" \
--tag "${DOCKER_IMAGE}:$(_get-version)" \
--build-arg BUILD_VERSION="$(_get-effective-version)" \
--tag "${DOCKER_IMAGE}:$(_get-effective-version)" \
--rm \
--file "${dir}/Dockerfile.effective" \
.
# Tag the image with all the stated versions, see the documentation for more details
for version in $(_get-all-versions) latest; do
docker tag "${DOCKER_IMAGE}:$(_get-version)" "${DOCKER_IMAGE}:${version}"
for version in $(_get-all-effective-versions) latest; do
docker tag "${DOCKER_IMAGE}:$(_get-effective-version)" "${DOCKER_IMAGE}:${version}"
done
docker rmi --force "$(docker images | grep "<none>" | awk '{print $3}')" 2> /dev/null ||:
}
Expand All @@ -59,7 +59,7 @@ function docker-check-test() {
# shellcheck disable=SC2086,SC2154
docker run --rm --platform linux/amd64 \
${args:-} \
"${DOCKER_IMAGE}:$(_get-version)" 2>/dev/null \
"${DOCKER_IMAGE}:$(_get-effective-version)" 2>/dev/null \
${cmd:-} \
| grep -q "${check}" && echo PASS || echo FAIL
}
Expand All @@ -75,7 +75,7 @@ function docker-run() {
# shellcheck disable=SC2086
docker run --rm --platform linux/amd64 \
${args:-} \
"${DOCKER_IMAGE}:$(dir="$dir" _get-version)" \
"${DOCKER_IMAGE}:$(dir="$dir" _get-effective-version)" \
${cmd:-}
}

Expand All @@ -86,7 +86,7 @@ function docker-push() {

local dir=${dir:-$PWD}
# Push all the image tags based on the stated versions, see the documentation for more details
for version in $(dir="$dir" _get-all-versions) latest; do
for version in $(dir="$dir" _get-all-effective-versions) latest; do
docker push "${DOCKER_IMAGE}:${version}"
done
}
Expand All @@ -97,7 +97,7 @@ function docker-push() {
function docker-clean() {

local dir=${dir:-$PWD}
for version in $(dir="$dir" _get-all-versions) latest; do
for version in $(dir="$dir" _get-all-effective-versions) latest; do
docker rmi "${DOCKER_IMAGE}:${version}" > /dev/null 2>&1 ||:
done
rm -f \
Expand All @@ -112,17 +112,19 @@ function docker-clean() {
function version-create-effective-file() {

local dir=${dir:-$PWD}
build_datetime=${BUILD_DATETIME:-$(date -u +'%Y-%m-%dT%H:%M:%S%z')}
if [ -f "$dir/VERSION" ]; then
local version_file="$dir/VERSION"
local build_datetime=${BUILD_DATETIME:-$(date -u +'%Y-%m-%dT%H:%M:%S%z')}

if [ -f "$version_file" ]; then
# shellcheck disable=SC2002
cat "$dir/VERSION" | \
sed "s/yyyy/$(date --date="${build_datetime}" -u +"%Y")/g" | \
sed "s/mm/$(date --date="${build_datetime}" -u +"%m")/g" | \
sed "s/dd/$(date --date="${build_datetime}" -u +"%d")/g" | \
sed "s/HH/$(date --date="${build_datetime}" -u +"%H")/g" | \
sed "s/MM/$(date --date="${build_datetime}" -u +"%M")/g" | \
sed "s/SS/$(date --date="${build_datetime}" -u +"%S")/g" | \
sed "s/hash/$(git rev-parse --short HEAD)/g" \
cat "$version_file" | \
sed "s/\(\${yyyy}\|\$yyyy\)/$(date --date="${build_datetime}" -u +"%Y")/g" | \
sed "s/\(\${mm}\|\$mm\)/$(date --date="${build_datetime}" -u +"%m")/g" | \
sed "s/\(\${dd}\|\$dd\)/$(date --date="${build_datetime}" -u +"%d")/g" | \
sed "s/\(\${HH}\|\$HH\)/$(date --date="${build_datetime}" -u +"%H")/g" | \
sed "s/\(\${MM}\|\$MM\)/$(date --date="${build_datetime}" -u +"%M")/g" | \
sed "s/\(\${SS}\|\$SS\)/$(date --date="${build_datetime}" -u +"%S")/g" | \
sed "s/\(\${hash}\|\$hash\)/$(git rev-parse --short HEAD)/g" \
> "$dir/.version"
fi
}
Expand Down Expand Up @@ -198,7 +200,10 @@ function _create-effective-dockerfile() {
function _replace-image-latest-by-specific-version() {

local dir=${dir:-$PWD}
versions_file=$(git rev-parse --show-toplevel)/.tool-versions
local versions_file=$(git rev-parse --show-toplevel)/.tool-versions
local dockerfile="${dir}/Dockerfile.effective"
local build_datetime=${BUILD_DATETIME:-$(date -u +'%Y-%m-%dT%H:%M:%S%z')}

if [ -f "$versions_file" ]; then
# First, list the entries specific for Docker to take precedence, then the rest
content=$(grep " docker/" "$versions_file"; grep -v " docker/" "$versions_file")
Expand All @@ -207,9 +212,26 @@ function _replace-image-latest-by-specific-version() {
line=$(echo "$line" | sed "s/^#\s*//; s/\s*#.*$//" | sed "s;docker/;;")
name=$(echo "$line" | awk '{print $1}')
version=$(echo "$line" | awk '{print $2}')
sed -i "s;FROM ${name}:latest;FROM ${name}:${version};g" "${dir}/Dockerfile.effective"
sed -i "s;\(FROM .*\) ${name}:latest;\1 ${name}:${version};g" "$dockerfile"
done
fi

if [ -f "$dockerfile" ]; then
# shellcheck disable=SC2002
cat "$dockerfile" | \
sed "s/\(\${yyyy}\|\$yyyy\)/$(date --date="${build_datetime}" -u +"%Y")/g" | \
sed "s/\(\${mm}\|\$mm\)/$(date --date="${build_datetime}" -u +"%m")/g" | \
sed "s/\(\${dd}\|\$dd\)/$(date --date="${build_datetime}" -u +"%d")/g" | \
sed "s/\(\${HH}\|\$HH\)/$(date --date="${build_datetime}" -u +"%H")/g" | \
sed "s/\(\${MM}\|\$MM\)/$(date --date="${build_datetime}" -u +"%M")/g" | \
sed "s/\(\${SS}\|\$SS\)/$(date --date="${build_datetime}" -u +"%S")/g" | \
sed "s/\(\${hash}\|\$hash\)/$(git rev-parse --short HEAD)/g" \
> "$dockerfile.tmp"
mv "$dockerfile.tmp" "$dockerfile"
fi

# Do not ignore the issue if 'latest' is used in the effective image
sed -Ei "/# hadolint ignore=DL3007$/d" "${dir}/Dockerfile.effective"
}

# Append metadata to the end of Dockerfile.
Expand All @@ -228,7 +250,7 @@ function _append-metadata() {
# Print top Docker image version.
# Arguments (provided as environment variables):
# dir=[path to the image directory where the Dockerfile is located, default is '.']
function _get-version() {
function _get-effective-version() {

local dir=${dir:-$PWD}
head -n 1 "${dir}/.version"
Expand All @@ -237,7 +259,7 @@ function _get-version() {
# Print all Docker image versions.
# Arguments (provided as environment variables):
# dir=[path to the image directory where the Dockerfile is located, default is '.']
function _get-all-versions() {
function _get-all-effective-versions() {

local dir=${dir:-$PWD}
cat "${dir}/.version"
Expand Down

0 comments on commit 2e26ba9

Please sign in to comment.