Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to register validating admission webhook (/queues/validate): the server could not find the requested resource #2336

Closed
SunMinghui19 opened this issue Jul 6, 2022 · 8 comments
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@SunMinghui19
Copy link

What happened:
The logs of the volcano-scheduler pod show this error:
···
Failed to register validating admission webhook (/queues/validate): the server could not find the requested resource
···

What you expected to happen:
Not see this error

How to reproduce it (as minimally and precisely as possible):
From the Volcano 1.5 release branch, install volcano using:

kubectl apply -f installer/volcano-development.yaml
and then look at the logs of the volcano-admission pod

kubectl -n volcano-system logs volcano-admission-7454db885f-5hsns

Anything else we need to know?:
Here's a more extensive output of the log (it repeats a few times in the log)
···
I0706 15:01:37.080830 1 server.go:78] Registered '/queues/validate' as webhook.
I0706 15:01:37.080833 1 server.go:81] Registered configuration for webhook </queues/validate>
I0706 15:01:37.080836 1 util.go:53] The service of webhook manager is <volcano-admission-service/volcano-system//queues/validate>.
I0706 15:01:37.081771 1 util.go:181] Creating ValidatingWebhookConfiguration &ValidatingWebhookConfiguration{ObjectMeta:{volcano-admission-service-queues-validate 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Webhooks:[]ValidatingWebhook{ValidatingWebhook{Name:validatequeue.volcano.sh,ClientConfig:WebhookClientConfig{Service:&ServiceReference{Namespace:volcano-system,Name:volcano-admission-service,Path:/queues/validate,Port:nil,},CABundle:[45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 68 84 122 67 67 65 106 101 103 65 119 73 66 65 103 73 85 85 106 122 114 117 102 81 82 103 85 81 69 86 100 103 90 97 71 78 47 81 117 84 70 43 83 69 119 68 81 89 74 75 111 90 73 104 118 99 78 65 81 69 76 10 66 81 65 119 78 122 69 49 77 68 77 71 65 49 85 69 65 119 119 115 100 109 57 115 89 50 70 117 98 121 49 104 90 71 49 112 99 51 78 112 98 50 52 116 99 50 86 121 100 109 108 106 90 83 53 50 98 50 120 106 89 87 53 118 10 76 88 78 53 99 51 82 108 98 83 53 122 100 109 77 119 72 104 99 78 77 106 73 119 78 122 65 50 77 84 85 119 77 84 77 49 87 104 99 78 77 122 73 119 78 122 65 122 77 84 85 119 77 84 77 49 87 106 65 51 77 84 85 119 10 77 119 89 68 86 81 81 68 68 67 120 50 98 50 120 106 89 87 53 118 76 87 70 107 98 87 108 122 99 50 108 118 98 105 49 122 90 88 74 50 97 87 78 108 76 110 90 118 98 71 78 104 98 109 56 116 99 51 108 122 100 71 86 116 10 76 110 78 50 89 122 67 67 65 83 73 119 68 81 89 74 75 111 90 73 104 118 99 78 65 81 69 66 66 81 65 68 103 103 69 80 65 68 67 67 65 81 111 67 103 103 69 66 65 76 76 57 99 88 99 87 47 97 116 106 49 102 114 113 10 102 53 80 43 75 65 121 89 103 51 51 122 86 43 90 105 89 108 116 72 84 55 99 47 110 48 113 107 102 76 65 49 72 120 113 55 75 76 110 109 112 114 74 82 113 78 121 68 54 83 87 65 72 72 106 69 86 72 80 82 70 98 73 110 10 83 72 72 85 117 114 65 102 68 72 86 75 77 47 109 54 97 79 110 103 66 99 66 78 114 120 55 50 110 97 75 68 90 106 82 84 68 88 99 98 102 121 75 109 111 87 50 110 49 104 110 69 57 113 65 114 54 72 87 80 121 83 89 105 10 81 86 56 76 52 109 113 118 74 85 102 105 122 104 102 111 77 50 52 102 48 119 77 73 100 103 114 49 110 113 107 73 90 55 98 57 57 84 120 84 107 115 89 54 105 75 117 116 73 87 121 65 74 88 57 118 79 52 48 69 115 52 51 68 10 83 52 52 107 105 120 55 114 98 106 107 87 72 112 81 104 79 119 74 101 71 47 70 88 109 77 109 116 108 115 103 76 75 116 100 83 87 87 68 80 121 43 53 116 48 114 80 74 85 107 52 72 67 83 117 89 102 67 76 70 115 105 100 106 10 88 75 107 122 89 114 83 119 82 114 120 50 48 102 112 81 56 75 108 72 101 116 121 114 112 74 117 89 101 52 57 71 105 73 88 67 102 66 72 103 117 77 66 103 100 119 57 66 66 113 79 100 55 117 87 43 90 65 117 103 70 116 88 82 10 113 122 69 50 88 81 99 67 65 119 69 65 65 97 78 84 77 70 69 119 72 81 89 68 86 82 48 79 66 66 89 69 70 65 116 106 99 122 97 103 120 117 67 112 75 97 98 82 107 51 83 119 81 70 117 79 122 65 74 113 77 66 56 71 10 65 49 85 100 73 119 81 89 77 66 97 65 70 65 116 106 99 122 97 103 120 117 67 112 75 97 98 82 107 51 83 119 81 70 117 79 122 65 74 113 77 65 56 71 65 49 85 100 69 119 69 66 47 119 81 70 77 65 77 66 65 102 56 119 10 68 81 89 74 75 111 90 73 104 118 99 78 65 81 69 76 66 81 65 68 103 103 69 66 65 69 90 76 80 83 78 99 79 90 74 70 78 80 50 67 111 113 106 111 99 54 122 122 78 105 84 54 43 83 55 87 88 121 68 119 116 98 54 110 10 90 56 109 66 82 51 117 118 66 55 109 114 82 122 76 67 116 107 76 72 108 67 109 78 110 101 85 53 74 118 97 110 75 81 43 116 113 88 79 68 107 105 98 67 107 69 88 71 66 57 111 56 103 102 48 54 73 52 70 88 112 118 102 119 10 57 119 51 121 86 108 87 57 47 101 101 87 113 107 104 73 116 106 103 100 119 68 78 120 81 57 117 107 66 47 103 89 55 86 106 71 97 104 100 98 110 80 47 111 57 70 81 54 121 84 116 98 56 103 65 81 99 109 121 97 106 75 80 85 10 75 118 121 104 107 88 49 56 50 49 107 119 78 100 105 105 99 68 98 100 110 52 80 66 119 112 108 72 69 55 109 120 48 100 76 120 89 109 87 84 43 102 67 50 102 74 97 74 74 56 122 101 86 100 114 73 83 65 100 101 109 84 77 121 10 43 79 54 78 112 69 56 90 73 87 109 83 107 78 85 76 110 106 81 80 72 114 83 83 74 73 74 57 68 120 73 100 51 100 88 110 105 116 81 72 68 113 72 97 76 115 70 105 74 86 68 89 80 118 50 90 107 120 100 103 109 119 115 49 10 65 71 88 120 53 82 73 107 56 76 82 43 70 114 49 57 74 113 48 121 43 103 79 105 115 84 86 105 70 84 113 87 78 50 110 108 88 116 90 78 119 102 116 55 106 69 73 61 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10],URL:nil,},Rules:[]RuleWithOperations{RuleWithOperations{Operations:[CREATE UPDATE DELETE],Rule:Rule{APIGroups:[scheduling.volcano.sh],APIVersions:[v1beta1],Resources:[queues],Scope:nil,},},},FailurePolicy:nil,NamespaceSelector:nil,SideEffects:nil,TimeoutSeconds:nil,AdmissionReviewVersions:[],MatchPolicy:nil,ObjectSelector:nil,},},}
E0706 15:01:37.082480 1 util.go:78] Failed to register validating admission webhook (/queues/validate): the server could not find the requested resource
···

Environment:

  • Volcano Version:1.5
  • Kubernetes version (use kubectl version): 1.23.0
  • Cloud provider or hardware configuration: 8C 8G M
  • OS (e.g. from /etc/os-release): ubuntu20.04
  • Kernel (e.g. uname -a):Linux k8s-master 5.4.0-33-generic Fix slice out of range error #37-Ubuntu SMP Thu May 21 12:53:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: kubeadm
  • Others:
@SunMinghui19 SunMinghui19 added the kind/bug Categorizes issue or PR as related to a bug. label Jul 6, 2022
@SunMinghui19
Copy link
Author

compose of volcano display here
image

I used the volcano-scheduler to deploy an MPI task, but the cluster had nothing to do. I don't use the command to acquire the MPI pod
image

I use the following commands to check the volcano's problem.
image
image

I ever used different K8s versions (example 1.15 1.16.8 etc)
I hope to solve the problem with community help, Thanks.

@william-wang
Copy link
Member

@SunMinghui19 The issue is similar to #2048 which is fixed by pr #2064. Please use a higer version 1.5.1 and above to have a try.

@SunMinghui19
Copy link
Author

@SunMinghui19 The issue is similar to #2048 which is fixed by pr #2064. Please use a higer version 1.5.1 and above to have a try.

Thanks for your answer, I delete the plugin version of 1.15, then, I redeploy a version of 1.16. but, the same problem occurred

@SunMinghui19
Copy link
Author

@SunMinghui19 The issue is similar to #2048 which is fixed by pr #2064. Please use a higer version 1.5.1 and above to have a try.

Thanks for your answer, I delete the plugin version of 1.15, then, I redeploy a version of 1.16. but, the same problem occurred

I suspected my Kubernets cluster was not healthy, I used many versions of Kubernetes like 1.15.1, 1.16.8,1.20.0,1.23.0, And in every version installed different plugins from1.0 to 1.5. The result of is identical, i am so depressed.

@william-wang
Copy link
Member

By the way, after you uninstalled old volcano plugin, did you delete the volcano webhookconfiguration manually?
You need to run kubectl get validatingwebhookconfigurations and kubectl get mutatingwebhookconfigurations to get the volcano webhookconfiguration and delete them and then install the new version.

@SunMinghui19
Copy link
Author

By the way, after you uninstalled old volcano plugin, did you delete the volcano webhookconfiguration manually? You need to run kubectl get validatingwebhookconfigurations and kubectl get mutatingwebhookconfigurations to get the volcano webhookconfiguration and delete them and then install the new version.

Thanks for your answer. I don't execute these commands, Now, I try it.

@SunMinghui19
Copy link
Author

SunMinghui19 commented Jul 7, 2022

image
I guess this execution is no sense, because the result is "No resources found"

@SunMinghui19
Copy link
Author

Thanks for your help, Because my execution error leads to unsuccessful, your solution is very accurate. After I upgrade the volcano version to 1.5.1, this error disappeared.

@william-wang william-wang self-assigned this Jul 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants