Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add helm chart with initial functional test #2

Merged
merged 66 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
0d5f6c1
change test path
scrungus Jun 26, 2024
aa0c566
Dockerfile
scrungus Jun 26, 2024
d2692e5
pin requirements
scrungus Jun 26, 2024
7eaa6f7
add python package conf
scrungus Jun 26, 2024
a0e584c
python version
scrungus Jun 26, 2024
3c483f7
initial CI setup
scrungus Jun 26, 2024
78e0a0b
initial helm charts
scrungus Jun 26, 2024
007e85d
add functional test
scrungus Jun 26, 2024
da9e10c
remove integration test for now
scrungus Jun 26, 2024
e2a35d7
rename file extensions
scrungus Jun 26, 2024
14f7310
add build and push
scrungus Jun 26, 2024
8f1b34f
use python3.10
scrungus Jun 26, 2024
e2c6a55
sign_image not valid
scrungus Jun 26, 2024
5390a1e
newline
scrungus Jun 26, 2024
42307f3
skip flake8 for now
scrungus Jun 26, 2024
bbd77b0
move to pyproject.toml
scrungus Jun 26, 2024
d8af137
specify python version
scrungus Jun 26, 2024
9a091ba
update permissions
scrungus Jun 26, 2024
ecbbd5d
move helpers
scrungus Jun 27, 2024
ee9ab4a
yaml indent
scrungus Jun 27, 2024
dfba34c
work out functional test steps manually
scrungus Jun 27, 2024
e1a91ae
change healthcheck endpoint
scrungus Jun 27, 2024
237d564
comment
scrungus Jun 27, 2024
b4b886c
permissions and accounts
scrungus Jun 27, 2024
4418050
copy in app dir
scrungus Jun 27, 2024
14d7ad9
manage.py location
scrungus Jun 27, 2024
bf29445
use settings.py for now
scrungus Jun 27, 2024
e97fd2b
update requirements
scrungus Jun 27, 2024
01532e1
database connection
scrungus Jun 28, 2024
85e9a74
pvc
scrungus Jul 1, 2024
ce2b123
close with
scrungus Jul 1, 2024
b959683
change pvc accessmode
scrungus Jul 1, 2024
60e4475
change probe port
scrungus Jul 1, 2024
ce3eece
add tzdata
scrungus Jul 1, 2024
446e9a6
turn debug off in prod, allowed hosts
scrungus Jul 1, 2024
a735da5
change to http port, add timeout
scrungus Jul 1, 2024
376966d
alllowed hosts
scrungus Jul 1, 2024
87afa69
add wait + status check
scrungus Jul 1, 2024
6570730
black
scrungus Jul 1, 2024
2905099
enable flake8
scrungus Jul 1, 2024
55a5bb2
ignore unit tests
scrungus Jul 1, 2024
a944a08
remove flake8 todo
scrungus Jul 1, 2024
3192bb5
comment out stestr
scrungus Jul 1, 2024
0bb90de
remove upterm
scrungus Jul 1, 2024
6307502
flake8
scrungus Jul 1, 2024
eb4e396
skip H301
scrungus Jul 1, 2024
41217cc
isort conf
scrungus Jul 1, 2024
5854156
ignore cover till we have unit tests
scrungus Jul 1, 2024
09b4504
wait for pod
scrungus Jul 1, 2024
a92aba6
wait for rollout
scrungus Jul 1, 2024
b8422f9
debug
scrungus Jul 1, 2024
a23c528
always run
scrungus Jul 1, 2024
684885e
don't exit on success
scrungus Jul 1, 2024
2b85057
readinessProbe
scrungus Jul 1, 2024
0a3151a
initial delay
scrungus Jul 1, 2024
d476f0b
sleep temp
scrungus Jul 1, 2024
9a9ce91
more robust
scrungus Jul 1, 2024
8c32c10
remove debug
scrungus Jul 1, 2024
6619597
remove sleep
scrungus Jul 1, 2024
78366ef
Add tilt component
assumptionsandg Jul 3, 2024
0cc4d46
django flexi settings
scrungus Jul 4, 2024
5c629fb
lint
scrungus Jul 4, 2024
1ae088f
variables + logs on functional test
scrungus Jul 4, 2024
82a8c4e
Merge pull request #4 from stackhpc/feature/tilt-component
scrungus Jul 8, 2024
ed66bae
remove clusterrole,serviceaccount
scrungus Jul 9, 2024
e1890fd
make database configurable, with default
scrungus Jul 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/functional.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Functional tests

on:
workflow_call:

jobs:
functional_test:
name: Operator functional tests via tox
timeout-minutes: 10
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Helm
uses: azure/setup-helm@v4
with:
version: v3.11.3

- name: Create k8s Kind Cluster
uses: helm/kind-action@v1.9.0

- name: Run test
timeout-minutes: 10
run: tools/functional_test.sh
25 changes: 25 additions & 0 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: on push to main

on:
push:
branches:
- main

concurrency:
group: main
cancel-in-progress: true

jobs:
unit_tests:
uses: ./.github/workflows/tox.yaml

publish_images:
uses: ./.github/workflows/publish-images.yaml

publish_charts:
needs: [publish_images]
uses: ./.github/workflows/publish-charts.yaml

functional_tests:
needs: [publish_images]
uses: ./.github/workflows/functional.yaml
38 changes: 38 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: on pull request

on:
pull_request:
types:
- opened
- synchronize
- ready_for_review
- edited
- reopened
branches:
- main

concurrency:
group: ${{ github.head_ref }}
cancel-in-progress: true

jobs:
# Run the unit tests on every PR, even from external repos
unit_tests:
uses: ./.github/workflows/tox.yaml

# When the PR is from a branch of the main repo, publish images and charts
publish_images:
uses: ./.github/workflows/publish-images.yaml
if: github.repository == 'stackhpc/coral-credits'

publish_charts:
needs: [publish_images]
uses: ./.github/workflows/publish-charts.yaml
if: github.repository == 'stackhpc/coral-credits'

# The functional tests require the runner image, so we can only run them
# once the image has been built, and on PRs from the main repo
functional_tests:
needs: [publish_images]
uses: ./.github/workflows/functional.yaml
if: github.repository == 'stackhpc/coral-credits'
33 changes: 33 additions & 0 deletions .github/workflows/publish-charts.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Publish charts

on:
workflow_call:
outputs:
chart-version:
description: The chart version that was published
value: ${{ jobs.publish_charts.outputs.chart-version }}

jobs:
publish_charts:
name: Publish Helm charts to GitHub pages
runs-on: ubuntu-latest
outputs:
chart-version: ${{ steps.semver.outputs.version }}
steps:
- name: Check out the repository
uses: actions/checkout@v4
with:
# This is important for the semver action to work correctly
# when determining the number of commits since the last tag
fetch-depth: 0

- name: Get SemVer version for current commit
id: semver
uses: stackhpc/github-actions/semver@master

- name: Publish Helm charts
uses: stackhpc/github-actions/helm-publish@master
with:
token: ${{ secrets.GITHUB_TOKEN }}
version: ${{ steps.semver.outputs.version }}
app-version: ${{ steps.semver.outputs.short-sha }}
File renamed without changes.
46 changes: 46 additions & 0 deletions .github/workflows/publish-images.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Publish images

on:
workflow_call:

jobs:
build_push_coral_credits_image:
name: Build and push coral credits image
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # needed for signing the images with GitHub OIDC Token
packages: write # required for pushing container images
security-events: write # required for pushing SARIF files

steps:
- name: Check out the repository
uses: actions/checkout@v4

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Calculate metadata for image
id: image-meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/stackhpc/coral-credits
# Produce the branch name or tag and the SHA as tags
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=

- name: Build and push image
uses: stackhpc/github-actions/docker-multiarch-build-push@master
with:
cache-key: coral-credits
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.image-meta.outputs.tags }}
labels: ${{ steps.image-meta.outputs.labels }}
13 changes: 13 additions & 0 deletions .github/workflows/tag.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: on tag

on:
push:
tags: ['**']

jobs:
publish_images:
uses: ./.github/workflows/publish-images.yaml

publish_charts:
needs: [publish_images]
uses: ./.github/workflows/publish-charts.yaml
38 changes: 38 additions & 0 deletions .github/workflows/tox.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Tox unit tests

on:
workflow_call:

jobs:
build:
name: Tox unit tests and linting
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10']

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install tox

- name: Test with tox
run: tox

# TODO(tylerchristie): add unit tests
# - name: Generate coverage reports
# run: tox -e cover

- name: Archive code coverage results
uses: actions/upload-artifact@v4
with:
name: code-coverage-report
path: cover/
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,4 @@ context


# don't check in django bits
coral_credits/settings.py
db.sqlite3
2 changes: 1 addition & 1 deletion .stestr.conf
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[DEFAULT]
test_path=./azimuth_caas_operator/tests
test_path=./coral-credits/tests
top_dir=./
62 changes: 62 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
FROM ubuntu:jammy as build-image

RUN apt-get update && \
apt-get upgrade -y && \
apt-get install --no-install-recommends python3.10-venv git -y && \
rm -rf /var/lib/apt/lists/*

# build into a venv we can copy across
RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"

COPY ./requirements.txt /coral-credits/requirements.txt
RUN pip install -U pip setuptools
RUN pip install --requirement /coral-credits/requirements.txt

# Django fails to load templates if this is installed the "regular" way
# If we use an editable mode install then it works
COPY . /coral-credits
RUN pip install --no-deps -e /coral-credits

#
# Now the image we run with
#
FROM ubuntu:jammy as run-image

RUN apt-get update && \
apt-get upgrade -y && \
apt-get install --no-install-recommends python3 tini ca-certificates -y && \
rm -rf /var/lib/apt/lists/*

# Copy across the venv
COPY --from=build-image /venv /venv
ENV PATH="/venv/bin:$PATH"

# Copy across the app
COPY --from=build-image /coral-credits /coral-credits

# Create the user that will be used to run the app
ENV APP_UID 1001
ENV APP_GID 1001
ENV APP_USER app
ENV APP_GROUP app
RUN groupadd --gid $APP_GID $APP_GROUP && \
useradd \
--no-create-home \
--no-user-group \
--gid $APP_GID \
--shell /sbin/nologin \
--uid $APP_UID \
$APP_USER

# Don't buffer stdout and stderr as it breaks realtime logging
ENV PYTHONUNBUFFERED 1

# TODO(tylerchristie): django flexi settings

# By default, serve the app on port 8080 using the app user
EXPOSE 8080
USER $APP_UID
ENTRYPOINT ["tini", "-g", "--"]
#TODO(tylerchristie): use gunicorn + wsgi like azimuth
CMD ["python", "/coral-credits/manage.py", "runserver", "0.0.0.0:8080"]
23 changes: 23 additions & 0 deletions charts/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
7 changes: 7 additions & 0 deletions charts/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v2
name: coral-credits
description: Helm chart for deploying the Coral Credits API.
type: application
# The version and appVersion are set by the CI script
version: 0.1.0
appVersion: "main"
53 changes: 53 additions & 0 deletions charts/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "coral-credits.name" -}}
{{- .Chart.Name | lower | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "coral-credits.fullname" -}}
{{- if contains .Chart.Name .Release.Name }}
{{- .Release.Name | lower | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name .Chart.Name | lower | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "coral-credits.chart" -}}
{{-
printf "%s-%s" .Chart.Name .Chart.Version |
replace "+" "_" |
trunc 63 |
trimSuffix "-" |
trimSuffix "." |
trimSuffix "_"
}}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "coral-credits.selectorLabels" -}}
app.kubernetes.io/name: {{ include "coral-credits.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "coral-credits.labels" -}}
helm.sh/chart: {{ include "coral-credits.chart" . }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
{{ include "coral-credits.selectorLabels" . }}
{{- end }}
Loading
Loading