Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Capacity plugin implement #3283

Merged
merged 4 commits into from
May 11, 2024
Merged

Capacity plugin implement #3283

merged 4 commits into from
May 11, 2024

Conversation

Monokaix
Copy link
Member

@Monokaix Monokaix commented Dec 28, 2023 •
edited
Loading

Third part of #3242.
This pr should be merged after volcano-sh/apis#121.

@volcano-sh-bot volcano-sh-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 28, 2023
@Monokaix Monokaix mentioned this pull request Dec 28, 2023
Closed
3 tasks
@Monokaix Monokaix changed the title Capacity plugin implement [WIP]Capacity plugin implement Dec 28, 2023
@volcano-sh-bot volcano-sh-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 28, 2023
lowang-bh
lowang-bh reviewed Dec 29, 2023
View reviewed changes
pkg/scheduler/actions/reclaim/reclaim.go
@@ -95,6 +95,10 @@ func (ra *Action) Execute(ssn *framework.Session) {
klog.V(3).Infof("Queue <%s> is overused, ignore it.", queue.Name)
continue
}
if !ssn.Preemptive(queue) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think it should consider the resource demension. Eg, queue is overused on GPU resource, but CPU/MEM is not overused and incomeing job just need cpu/mem, we can still reclaim from other queues to get CPU/MEM back.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think it should consider the resource demension. Eg, queue is overused on GPU resource, but CPU/MEM is not overused and incomeing job just need cpu/mem, we can still reclaim from other queues to get CPU/MEM back.

Updated.

pkg/scheduler/plugins/capacity/capacity.go Outdated
queue := obj.(*api.QueueInfo)
attr := cp.queueOpts[queue.UID]

canReclaim := !attr.deserved.LessEqual(attr.allocated, api.Zero)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mybe all dimentions resource is large than deserved can be regarded as not reclaimable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah，you are right.

@Monokaix Monokaix force-pushed the capacity-test branch 2 times, most recently from 8965847 to 24f226f Compare January 18, 2024 07:48
@volcano-sh-bot volcano-sh-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 9, 2024
@Monokaix Monokaix force-pushed the capacity-test branch 3 times, most recently from 9fdf693 to db865c2 Compare May 9, 2024 13:51
@lowang-bh
Copy link
Member

I am a litte confused.

The Guarantee can be treated as min resource which is the guaranteed resource of a queue, and can be configured by user.
and capacity is the max resource a tenant can use.

If a user wants a configurable min resource, Guarantee is ok, just disable proportion plugin and enable this capacity plugin(I think they can not be used at the same time, right?).

So Why adding another field Deserved to queue spec in volcano-sh/apis#121.

And what's more, the meaning of Deserved is not consistent with the comments This part of resource can be shared with other queues and reclaimed back

If it is a field representing a resource can be shared with other queues and reclaimed back, elastic or other word is more proper.

BTW, this plugin can not be used with proportion plugin, am I right?

@Monokaix
Copy link
Member Author

I am a litte confused.

The Guarantee can be treated as min resource which is the guaranteed resource of a queue, and can be configured by user. and capacity is the max resource a tenant can use.

If a user wants a configurable min resource, Guarantee is ok, just disable proportion plugin and enable this capacity plugin(I think they can not be used at the same time, right?).

So Why adding another field Deserved to queue spec in volcano-sh/apis#121.

And what's more, the meaning of Deserved is not consistent with the comments This part of resource can be shared with other queues and reclaimed back

If it is a field representing a resource can be shared with other queues and reclaimed back, elastic or other word is more proper.

BTW, this plugin can not be used with proportion plugin, am I right?

It's a replace of proportion plugin and can not use with proportion plugin at the same time, proportion plugin divided resources by weight while capacity plugin divided resources directly by deserved, which is more intuitive, you can learn more in design doc https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-scheduling.md.

@lowang-bh
Copy link
Member

t's a replace of proportion plugin and can not use with proportion plugin at the same time, proportion plugin divided resources by weight while capacity plugin divided resources directly by deserved, which is more intuitive, you can learn more in design doc https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-scheduling.md.

I see it. thanks.

@Monokaix Monokaix force-pushed the capacity-test branch 4 times, most recently from 74c06da to e0ff474 Compare May 11, 2024 02:33
@Monokaix
capacity plugin implementation
e8dfb78 
Signed-off-by: Xuzheng Chang <changxuzheng@huawei.com>
@Monokaix
Add capacity plugin e2e test
062d39c 
Signed-off-by: Xuzheng Chang <changxuzheng@huawei.com>
@Monokaix
make generate-yaml to update crd
cf39928 
Signed-off-by: Monokaix <changxuzheng@huawei.com>
@Monokaix
Add user guide for capacity plugin
262c7b2 
Signed-off-by: Xuzheng Chang <changxuzheng@huawei.com>
@Monokaix Monokaix changed the title [WIP]Capacity plugin implement Capacity plugin implement May 11, 2024
@volcano-sh-bot volcano-sh-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 11, 2024
william-wang
william-wang approved these changes May 11, 2024
View reviewed changes
Copy link
Member

@william-wang william-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@volcano-sh-bot volcano-sh-bot added the lgtm Indicates that a PR is ready to be merged. label May 11, 2024
@volcano-sh-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: william-wang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@volcano-sh-bot volcano-sh-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 11, 2024
@volcano-sh-bot volcano-sh-bot merged commit 703745a into volcano-sh:master May 11, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lowang-bh lowang-bh lowang-bh left review comments

@william-wang william-wang william-wang approved these changes

@k82cn k82cn Awaiting requested review from k82cn

@merryzhou merryzhou Awaiting requested review from merryzhou

@Thor-wl Thor-wl Awaiting requested review from Thor-wl

Assignees

@william-wang william-wang

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Monokaix @lowang-bh @volcano-sh-bot @william-wang