Set webhook failurePolicy to Ignore on controller pod shutdown #177
Conversation
pkg/utils/shutdown.go
Outdated
|
|
||
| func Shutdown() { | ||
| updateFinalizers() | ||
| //updateWebhooks() |
There was a problem hiding this comment.
Shall it be commented out?
pkg/utils/shutdown.go
Outdated
| shutdownLog.Info("Clearing finalizers on exit") | ||
| networkList := &sriovnetworkv1.SriovNetworkList{} | ||
| err := c.List(context.TODO(), networkList) | ||
| c, err := dynamic.NewForConfig(getConfig()) |
There was a problem hiding this comment.
Can we use the GetConfigOrDie from "sigs.k8s.io/controller-runtime/pkg/client/config" instead getConfig()? So that we can support running the operator in both in-cluster and local mode. The local mode is quite handy for development.
pkg/utils/shutdown.go
Outdated
| if err != nil { | ||
| shutdownLog.Error(err, "Error creating client") | ||
| } | ||
| sriovNetworkClient := c.Resource(sriovnetworksGVR) |
There was a problem hiding this comment.
There is a go-client implementation under package/client for the SriovNetwork CRDs, I think we can use it here as well.
There was a problem hiding this comment.
Thanks for this tip
Done
pkg/utils/shutdown.go
Outdated
| if err != nil { | ||
| shutdownLog.Error(err, "Error getting client") | ||
| } | ||
| validatingWebhookClient := clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations() |
There was a problem hiding this comment.
Shall we also create a updateValidatingWebhooks function as the updateMutatingWebhooks function bellow?
pkg/utils/shutdown.go
Outdated
| shutdownLog.Info("Clearing finalizers on exit") | ||
| networkList := &sriovnetworkv1.SriovNetworkList{} | ||
| err := c.List(context.TODO(), networkList) | ||
| //c2, err := dynamic.NewForConfig(conf.GetConfigOrDie()) |
There was a problem hiding this comment.
Shall this line be removed?
There was a problem hiding this comment.
removed, sorry for that one
This PR sets the failurePolicy of sriov webhooks to Ignore when an operator pod is exiting. This ensures that the policy does not cause incoming resources to fail when the sriov pods are down. The policy is changed to back to Fail on the next sriovconfig reconciliation loop (for example when the controller pod is started again).
The PR also changes the kubernetes API used in the shutdown function from the controller-runtime api to the basic client-go api.
This is due issues with the controller api during the controller shutdown. It was noticed that the api calls sometimes failed due to Informer not able to sync, probably caused by the manager being closed before the shutdown function was invoked.