Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document labels, annotations and taints for JobSet #47383

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions content/en/docs/reference/labels-annotations-taints/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2332,6 +2332,114 @@ Starting in v1.16, this annotation was removed in favor of
[Pod Priority](/docs/concepts/scheduling-eviction/pod-priority-preemption/).
{{< /note >}}

### jobset.sigs.k8s.io/jobset-name

Type: Label
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these are used as both labels and annotations (except the taint).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay i will register for both labels and annotation


Example: `jobset.sigs.k8s.io/jobset-name: "my-jobset"`

Used on: Jobs, Pods

This label is used to store the name of the JobSet to which a Job or Pod belongs.
JobSet is an extension API that you can deploy into your Kubernetes cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the word "JobSet" a link to the JobSet github repo for readers who are unfamiliar with it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure I will link that.


### jobset.sigs.k8s.io/replicatedjob-replicas

Type: Annotation

Example: `jobset.sigs.k8s.io/replicatedjob-replicas: "5"`

Used on: ReplicatedJobs

This annotation specifies the number of replicas for a ReplicatedJob.
sftim marked this conversation as resolved.
Show resolved Hide resolved
ReplicatedJob is an extension API that you can deploy into your Kubernetes cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this sentence, it is not accurate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay i will remove it


### jobset.sigs.k8s.io/replicatedjob-name

Type: Label

Example: `jobset.sigs.k8s.io/replicatedjob-name: "my-replicatedjob"`

Used on: Jobs

This label is used to index into a Job's labels and retrieve the name of the parent ReplicatedJob.
ReplicatedJob is an extension API that you can deploy into your Kubernetes cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this sentence please, ReplicatedJob is not an extension API that can be installed on your cluster, that is JobSet :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh sorry, i missed it !!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed it. PTAL!!


### jobset.sigs.k8s.io/job-index

Type: Annotation

Example: `jobset.sigs.k8s.io/job-index: "0"`

Used on: Jobs

This annotation stores the index of the Job within the ReplicatedJob.
ReplicatedJob is an extension API that you can deploy into your Kubernetes cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This annotation stores the index of the Job within the ReplicatedJob.
ReplicatedJob is an extension API that you can deploy into your Kubernetes cluster.
This label/annotation is set by the JobSet controller on child Jobs and Pods. It contains the index of the Job replica within its parent ReplicatedJob.


### jobset.sigs.k8s.io/job-key

Type: Label
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also used as annotation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay i will register this as annotation also


Example: `jobset.sigs.k8s.io/job-key: "my-job-key"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace "my-job-key" with a SHA256 hash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure l will replace it


Used on: Jobs

This label provides a unique key for the job.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This label provides a unique key for the job.
This label/annotation is set by the JobSet controller on child Jobs and Pods of a JobSet. The value will be the SHA256 hash of the namespaced Job name.


### alpha.jobset.sigs.k8s.io/exclusive-topology

Type: Annotation

Example: `alpha.jobset.sigs.k8s.io/exclusive-topology: "zone"`

Used on: JobSets, ReplicatedJobs

You can set this annotation on a JobSet or a ReplicatedJob template to ensure exclusive job placement per topology group.
JobSet and ReplicaSet are extension APIs that you can deploy into your Kubernetes cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this sentence and make the word JobSet a link to the jobset repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure


### alpha.jobset.sigs.k8s.io/node-selector

Type: Annotation

Example: `alpha.jobset.sigs.k8s.io/node-selector-strategy=true`

Used on: JObs ,Pods
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Used on: JObs ,Pods
Used on: Jobs ,Pods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uff my bad!! i will fix it.


This annotation can be applied to a JobSet. When it's set, the JobSet controller modifies the Jobs and their corresponding Pods by adding node selectors. This ensures exclusive job placement per topology domain, restricting the scheduling of these Pods to specific nodes based on the strategy.

### alpha.jobset.sigs.k8s.io/namespaced-job

Type: Annotation, Label

Example: `alpha.jobset.sigs.k8s.io/namespaced-job:my-jobset-replicatedjob-A-0`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example: `alpha.jobset.sigs.k8s.io/namespaced-job:my-jobset-replicatedjob-A-0`
Example: `alpha.jobset.sigs.k8s.io/default_myjobset-replicatedjob-0`

see https://github.com/kubernetes-sigs/jobset/blob/main/pkg/controllers/jobset_controller.go#L804

Also, the format of child job names for JobSets is: {jobset}-{replicatedJob}-{jobIndex}


Used on: Jobs, Pods

The JobSet controller adds this label/annotation to Jobs and Pods that are part of a JobSet. It stores the namespaced Job name, which is used for logging and event management within the JobSet context.

### alpha.jobset.sigs.k8s.io/no-schedule

Type: Taint

Example: `alpha.jobset.sigs.k8s.io/no-schedule: NoSchedule`

Used on: Nodes

The JobSet controller uses this taint to support its node labeling exclusive placement strategy.
JobSet is an extension API that you can deploy into your Kubernetes cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice here you mention "JobSet is an extension API that you can deploy into your Kubernetes cluster" but not in any other taint/label/annotation descriptions.

I think it would make more sense to delete this and instead make the word "JobSet" a link to https://github.com/kubernetes-sigs/jobset

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure i will add link for Jobset


### jobset.sigs.k8s.io/coordinator

Type: Annotation, Label
Adarsh-verma-14 marked this conversation as resolved.
Show resolved Hide resolved

Example: `jobset.sigs.k8s.io/coordinator: "coordinator-endpoint"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example: `jobset.sigs.k8s.io/coordinator: "coordinator-endpoint"`
Example: `jobset.sigs.k8s.io/coordinator: "myjobset-workers-0-0.headless-svc"`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay i will replace with endpoint.


Used on: Jobs, Pods

This annotation/label is used on Jobs and Pods to store a stable network endpoint where the coordinator
pod can be reached if the JobSet spec defines the `.spec.coordinator` field.
JobSet is an extension API that you can deploy into your Kubernetes cluster.

## Annotations used for audit

<!-- sorted by annotation -->
Expand Down