Add support for podGroup number limits for one queue

[hex108]

What this PR does / why we need it:
Add support for podGroup number limits for one queue

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #452

Special notes for your reviewer:
The PR add an admission webhook server. It is a start for other features, e.g. set default queue for each podgroup, set scheduler name based on namespace annotations. I'll add them after it is merged.

Release note:

Add support for podGroup number limits for one queue

[hex108]

/assign @k82cn

[hex108]

I have manually tested the feature. It works. I'll add doc and e2e tests in follow up PR.

The steps to make it work:

• Build the image

  $ make images

• Install kube-batch

  $ helm install $GOPATH/src/github.com/kubernetes-sigs/kube-batch/deployment/kube-batch --namespace kube-system

• Install the webhook server

  $ helm install $GOPATH/src/github.com/kubernetes-sigs/kube-batch/deployment/webhook-server --namespace kube-system

• Create following Queue

  $ cat queue.yaml
   apiVersion: scheduling.incubator.k8s.io/v1alpha1
   kind: Queue
   metadata:
        name: test-limit
   spec:
       weight: 2
       maxPodGroupNumber: 2
   $ kubectl create -f ./queue.yaml
  

  It will create a queue that limits the number of PodGroup in it to 2.

• Try to create 3 PodGroups in the above Queue

  $ cat podgroup1.yaml
  apiVersion: scheduling.incubator.k8s.io/v1alpha1
  kind: PodGroup
  metadata:
       name: pg1
  spec:
      queue: test-limit
      minMember: 3
   $ kubect create -f ./podgroup1.yaml
   podgroup.scheduling.incubator.k8s.io/pg1 created
  

   $ cat podgroup2.yaml
   apiVersion: scheduling.incubator.k8s.io/v1alpha1
   kind: PodGroup
   metadata:
        name: pg2
   spec:
       queue: test-limit
       minMember: 3
    $ kubect create -f ./podgroup2.yaml
    podgroup.scheduling.incubator.k8s.io/pg2 created
  

  $ cat podgroup3.yaml
  apiVersion: scheduling.incubator.k8s.io/v1alpha1
  kind: PodGroup
  metadata:
      name: pg3
  spec:
     queue: test-limit
     minMember: 3
  $ kubect create -f ./podgroup3.yaml
  Error from server: error when creating "./podgroup3.yaml": admission webhook "webhook-server.kube-system.svc" denied the request: exceed MaxPodGroupNumber
  

  As noticed, we created PodGroup pg1 and pg2 successfully, but failed to create pg3. The feaure works!

[k82cn]

We may handle this in a similar way to Quota which need have Queue controller firstly; as we also support normal k8s object in kube-batch which belongs to default queue.

[hex108]

We may handle this in a similar way to Quota which need have Queue controller firstly; as we also support normal k8s object in kube-batch which belongs to default queue.

What is Queue controller's responsibility?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hex108

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hex108]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hex108]

@k82cn PTAL

[k8s-ci-robot]

@hex108: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[hex108]

Close it since it is not needed.

/close

[k8s-ci-robot]

@hex108: Closed this PR.

In response to this:

Close it since it is not needed.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.