Added description of radix batch events. Updated job swagger (#247)

* Added description of radix batch events. Updated job swagger * Update public-site/docs/src/guides/jobs/notifications.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/guides/jobs/configure-jobs.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/guides/jobs/notifications.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/guides/jobs/notifications.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/guides/jobs/notifications.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/references/reference-radix-config/index.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/references/reference-radix-config/index.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> * Update public-site/docs/src/references/reference-radix-config/index.md Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com> --------- Co-authored-by: Fredrik Hatletvedt <32248439+Pespiri@users.noreply.github.com>
equinor · Oct 4, 2023 · ee592f7 · ee592f7
1 parent 6b519eb
commit ee592f7
Show file tree

Hide file tree

Showing 8 changed files with 174 additions and 11 deletions.
diff --git a/public-site/docs/src/.vuepress/config.ts b/public-site/docs/src/.vuepress/config.ts
@@ -78,6 +78,7 @@ export default defineUserConfig({
               children: [
                 '/guides/jobs/configure-jobs',
                 '/guides/jobs/job-manager-and-job-api',
+                '/guides/jobs/notifications',
                 '/guides/jobs/environment-variables',
                 '/guides/jobs/jobs-in-web-console',
                 '/guides/jobs/openapi-swagger'

diff --git a/public-site/docs/src/guides/jobs/configure-jobs.md b/public-site/docs/src/guides/jobs/configure-jobs.md
@@ -30,6 +30,8 @@ spec:
       schedulerPort: 9000
       timeLimitSeconds: 100
       backoffLimit: 5
+      notifications:
+        webhook: http://api:8080/monitor-batch-status
       resources:
         requests:
           memory: "256Mi"
@@ -58,6 +60,7 @@ Jobs have three extra configuration options; `schedulerPort`, `payload` and `tim
 - `node` (optional) defines gpu node requested for a job.
 - `timeLimitSeconds` (optional) defines maximum running time for a job.
 - `backoffLimit` (optional) defines the number of times a job will be restarted if its container exits in error.
+- `notifications.webhook` (optional) the Radix application component or job component endpoint, where Radix batch events will be posted when any of its job-component's running jobs or batches changes states.
 
 ### schedulerPort
 

diff --git a/public-site/docs/src/guides/jobs/environment-variables.md b/public-site/docs/src/guides/jobs/environment-variables.md
@@ -1,6 +1,6 @@
 ---
 title: Environment variables
-prev: job-manager-and-job-api
+prev: notifications
 next: jobs-in-web-console
 ---
 

diff --git a/public-site/docs/src/guides/jobs/index.md b/public-site/docs/src/guides/jobs/index.md
@@ -20,6 +20,7 @@ A max of 10 single completed jobs remains in the history of a job component. Whe
 
 * [Configure jobs](./configure-jobs.md)
 * [Job manager and job API](./job-manager-and-job-api.md)
+* [Notifications](./notifications.md)
 * [Environment variables](./environment-variables.md)
 * [Manage jobs in Radix web-console](./jobs-in-web-console.md)
 * [OpenAPI/Swagger](./openapi-swagger.md)

diff --git a/public-site/docs/src/guides/jobs/job-manager-and-job-api.md b/public-site/docs/src/guides/jobs/job-manager-and-job-api.md
@@ -1,7 +1,7 @@
 ---
 title: Job Manager and job API
 prev: configure-jobs
-next: environment-variables
+next: notifications
 ---
 
 ## Job Manager and job API

diff --git a/public-site/docs/src/guides/jobs/notifications.md b/public-site/docs/src/guides/jobs/notifications.md
@@ -0,0 +1,69 @@
+---
+title: Notifications
+prev: job-manager-and-job-api
+next: environment-variables
+---
+
+## Job notifications
+
+When job and batch status needs to be monitored, their statuses can be [pulled](./#get-a-state-of-a-batch). Another option is to use job component [notifications](../../references/reference-radix-config/#notifications), posted to one of Radix application components.
+
+The URL to the Radix application component or job component endpoint can be specified in the `webhook` option in the `notifications` section in the [radixconfig.yaml](../../references/reference-radix-config/). This is an endpoint where a Radix batch events will be posted when any of the running jobs or batches for this job component changes states. Notification about changes are sent by a `POST` method with an `application/json` `ContentType` with a [Radix batch event format](#radix-batch-event), which extends the [Radix batch status](./job-manager-and-job-api/#get-a-state-of-a-batch) format with a field `event`:
+* `Create` - this event is sent when a new Radix batch or job is created.
+* `Update` - this event is sent when an existing Radix batch or one of its jobs, including single job, has changed state.
+* `Delete` - this event is sent when an existing Radix batch or job is deleted.
+
+`notifications` and `webhook` can be specified on a job component configuration level and/or on `environmentConfig` level. Property in the `environmentConfig` will override those on the component level, if present.
+
+::: tip
+* Only a Radix application component or job component name and their ports can be used in the webhook URL
+* Only private ports of the specified component can be used, public ports cannot be used for this purpose.
+```yaml
+  components:
+    - name: frontend
+      src: frontend
+      ports:
+        - name: http
+          port: 8001
+        - name: job-monitoring
+          port: 8002
+      publicPort: http
+  jobs:
+    - name: compute
+      schedulerPort: 8080
+      notifications:
+        webhook: http://frontend:8002
+
+```
+:::
+
+### Radix batch event
+```json
+{
+  "name": "batch-compute-20220302155333-hrwl53mw",
+  "created": "2022-03-02T15:53:33+01:00",
+  "started": "2022-03-02T15:53:33+01:00",
+  "ended": "2022-03-02T15:54:00+01:00",
+  "status": "Succeeded",
+  "event": "Update",
+  "jobStatuses": [
+    {
+      "jobId": "job1",
+      "batchName": "batch-compute-20220302155333-hrwl53mw",
+      "name": "batch-compute-20220302155333-hrwl53mw-fjhcqwj7",
+      "created": "2022-03-02T15:53:36+01:00",
+      "started": "2022-03-02T15:53:36+01:00",
+      "ended": "2022-03-02T15:53:56+01:00",
+      "status": "Succeeded"
+    },
+    {
+      "jobId": "job2",
+      "batchName": "batch-compute-20220302155333-hrwl53mw",
+      "name": "batch-compute-20220302155333-hrwl53mw-qjzykhrd",
+      "created": "2022-03-02T15:53:39+01:00",
+      "started": "2022-03-02T15:53:39+01:00",
+      "status": "Running"
+    }
+  ]
+}
+```
diff --git a/public-site/docs/src/guides/jobs/swagger.json b/public-site/docs/src/guides/jobs/swagger.json
@@ -492,6 +492,92 @@
     }
   },
   "definitions": {
+    "BatchEvent": {
+      "description": "BatchEvent holds general information about batch event on change of status",
+      "type": "object",
+      "required": [
+        "name",
+        "created",
+        "event"
+      ],
+      "properties": {
+        "batchName": {
+          "description": "BatchName Optional Batch ID of a job",
+          "type": "string",
+          "x-go-name": "BatchName",
+          "example": "'batch1'"
+        },
+        "batchType": {
+          "description": "BatchType Single job or multiple jobs batch",
+          "type": "string",
+          "x-go-name": "BatchType",
+          "example": "\"job\""
+        },
+        "created": {
+          "description": "Created timestamp",
+          "type": "string",
+          "x-go-name": "Created",
+          "example": "2006-01-02T15:04:05Z"
+        },
+        "ended": {
+          "description": "Ended timestamp",
+          "type": "string",
+          "x-go-name": "Ended",
+          "example": "2006-01-02T15:04:05Z"
+        },
+        "event": {
+          "$ref": "#/definitions/Event"
+        },
+        "jobId": {
+          "description": "JobId Optional ID of a job",
+          "type": "string",
+          "x-go-name": "JobId",
+          "example": "'job1'"
+        },
+        "jobStatuses": {
+          "description": "JobStatuses of the jobs in the batch",
+          "type": "array",
+          "items": {
+            "$ref": "#/definitions/JobStatus"
+          },
+          "x-go-name": "JobStatuses"
+        },
+        "message": {
+          "description": "Message, if any, of the job",
+          "type": "string",
+          "x-go-name": "Message",
+          "example": "\"Error occurred\""
+        },
+        "name": {
+          "description": "Name of the job",
+          "type": "string",
+          "x-go-name": "Name",
+          "example": "calculator"
+        },
+        "started": {
+          "description": "Started timestamp",
+          "type": "string",
+          "x-go-name": "Started",
+          "example": "2006-01-02T15:04:05Z"
+        },
+        "status": {
+          "description": "Status of the job",
+          "type": "string",
+          "enum": [
+            "Waiting",
+            "Running",
+            "Succeeded",
+            "Stopping",
+            "Stopped",
+            "Failed",
+            "DeadlineExceeded"
+          ],
+          "x-go-name": "Status",
+          "example": "Waiting"
+        }
+      },
+      "x-go-package": "github.com/equinor/radix-job-scheduler/models/v1/events"
+    },
     "BatchScheduleDescription": {
       "description": "BatchScheduleDescription holds description about batch scheduling job",
       "type": "object",
@@ -560,7 +646,7 @@
           "x-go-name": "JobStatuses"
         },
         "message": {
-          "description": "Status message, if any, of the job",
+          "description": "Message, if any, of the job",
           "type": "string",
           "x-go-name": "Message",
           "example": "\"Error occurred\""
@@ -595,6 +681,10 @@
       },
       "x-go-package": "github.com/equinor/radix-job-scheduler/models/v1"
     },
+    "Event": {
+      "type": "string",
+      "x-go-package": "github.com/equinor/radix-job-scheduler/models/v1/events"
+    },
     "JobScheduleDescription": {
       "description": "JobScheduleDescription holds description about scheduling job",
       "type": "object",
@@ -670,7 +760,7 @@
           "example": "'job1'"
         },
         "message": {
-          "description": "Status message, if any, of the job",
+          "description": "Message, if any, of the job",
           "type": "string",
           "x-go-name": "Message",
           "example": "\"Error occurred\""

diff --git a/public-site/docs/src/references/reference-radix-config/index.md b/public-site/docs/src/references/reference-radix-config/index.md
@@ -55,7 +55,11 @@ The `build` section of the spec contains configuration needed during build (CI p
 `useBuildKit` - (optional, default `false`) build a component with Docker BuildKit. Read  [more](../../guides/build-secrets/#build-secrets-with-buildkit) in the guide.
 
 :::tip
-When an option `useBuildKit` is `true`, Radix uses [buildah](https://www.redhat.com/en/topics/containers/what-is-buildah) to build components. Buildah requires the `Dockerfile` instruction `FROM` to have a repository prefixing the docker image name.
+When the option `useBuildKit` is set to `true`, Radix will use [buildah](https://www.redhat.com/en/topics/containers/what-is-buildah) to build the components. Buildah requires the `Dockerfile` instruction `FROM` to have a repository prefixing the docker image name.
+Otherwise there will be an error during the docker image build:
+```bash
+Error: creating build container: short-name resolution enforced but cannot prompt without a TTY
+```
 
 E.g. instead of `FROM alpine` use `FROM docker.io/alpine`, as this `alpine` image is located in the [Docker Hub](https://hub.docker.com/) repository.
 :::
@@ -712,12 +716,7 @@ spec:
         webhook: http://api:8080/monitor-batch-status
 ```
 
-`webhook` is an optional URL to the Radix application component or job component, which will be called when any of this job-component running job or batch states are changed. Only changes are sent by POST method with a `application/json` `ContentType` in a [state of a batch format](../../guides/jobs/#get-a-state-of-a-batch).
-
-`notifications` and `webhook` can be specified on a job component configuration level and/or on `environmentConfig` level. Property in the `environmentConfig` will override those on the component level, if present.
-
-> * Only a Radix application component or job component name can be used as a domain in the URL
-> * Only private port of the specified component can be used. If this component has only public port - a second port need to be added and used
+`webhook` is an optional URL to the Radix application component or job component which will be called when any of the job-component's running jobs or batches changes states. Only changes are sent by POST method with a `application/json` `ContentType` in a [batch event format](../../guides/jobs/#get-a-state-of-a-batch).
 
 ### `monitoringConfig`