Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: fix Kubernetes CI Tests #413

Merged
merged 8 commits into from
Jul 28, 2021
Merged

ci: fix Kubernetes CI Tests #413

merged 8 commits into from
Jul 28, 2021

Conversation

aktech
Copy link
Contributor

@aktech aktech commented Jul 28, 2021

Fixes #412
and probably #397 #405

  • Fixes the image import to k3d, which was causing tests to not run.
  • Fix installation in k8s step
  • Update dask and distributed versions to remove version mismatch to avoid TLS handshake issues in the tests
  • Adds worker threads to KubeClusterConfig to avoid KeyError: 'worker_threads'
  • Misc - remove todos, copyright

cc @martindurant

@@ -126,6 +126,9 @@ gateway:
request: null
limit: null

# Number of threads available for a worker
threads: 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please explain this? It seems to me reasonable that in real use, multiple threads per worker can be a good thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a default value for testing, otherwise it would come out to be 0:

The value of the 'worker_threads' trait of a KubeClusterConfig instance should not be less than 1, but a value of 0 was specified.

In practice you'll override this with your own values, with whatever your requirement is

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that's fine.
Can you change the default to also be 1?

Copy link
Member

@jcrist jcrist Jul 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, this looks like a bug actually. By default the worker should use as many threads as cores (which has a pre-configured default). A user shouldn't be forced to configure the number of threads unless they want it to differ from the number of cores. I think we want worker_threads to set allow_none=True, then fix the logic in the k8s backend to use worker_cores if worker_threads is None.

This was added since the last release, so it hasn't gone out yet. Bug introduced in #353.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the input @jcrist . Let's make that a separate issue, then.

@martindurant
Copy link
Member

Any idea why the CI sin't running here?

@martindurant
Copy link
Member

(ah, you got there!)

@martindurant martindurant merged commit de75496 into dask:main Jul 28, 2021
@aktech aktech deleted the fix-k8s-tests branch July 28, 2021 15:44
@jacobtomlinson
Copy link
Member

🎉

@consideRatio consideRatio changed the title Fix Kubernetes CI Tests ci: fix Kubernetes CI Tests Apr 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failing Kubernetes Tests in the CI
5 participants