Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helm chart: expose worker_threads configuration #397

Closed
Kirill888 opened this issue May 27, 2021 · 1 comment
Closed

Helm chart: expose worker_threads configuration #397

Kirill888 opened this issue May 27, 2021 · 1 comment

Comments

@Kirill888
Copy link
Contributor

I would like to configure number of worker threads separately from worker.cores.{request|limit}.

But generated code here:

# Forward dask cluster configuration
for field, prop_name in [
# Scheduler config
("scheduler_cores", "scheduler.cores.request"),
("scheduler_cores_limit", "scheduler.cores.limit"),
("scheduler_memory", "scheduler.memory.request"),
("scheduler_memory_limit", "scheduler.memory.limit"),
("scheduler_extra_container_config", "scheduler.extraContainerConfig"),
("scheduler_extra_pod_config", "scheduler.extraPodConfig"),
# Worker config
("worker_cores", "worker.cores.request"),
("worker_cores_limit", "worker.cores.limit"),
("worker_memory", "worker.memory.request"),
("worker_memory_limit", "worker.memory.limit"),
("worker_extra_container_config", "worker.extraContainerConfig"),
("worker_extra_pod_config", "worker.extraPodConfig"),
# Additional fields
("image_pull_policy", "image.pullPolicy"),
("environment", "environment"),
("namespace", "namespace"),
]:

does not expose worker_threads configuration option. I think this should be easy to add?

I'd like to run 8 threads on an 8 core node, but I need to request fewer cores from k8s to fit pod into the node, with the current helm we end up with 7 threads worker on an 8 core instance and I don't think k8s needs one whole core for service processes.

@aktech
Copy link
Contributor

aktech commented Jul 30, 2021

I believe this was added in #413

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants