Ingress stil trying to add nodes to instance groups when using NEGs

[tlbdk]

We are using ingress-nginx with a manual load balancer in front but are looking to migrate to GCE ingress with NEG and would like to run both for a while. In theory this should be possible as there should be no reason for the GCE ingress to add the nodes to a load-balancer instance group when using NEG. But it seems to do this anyway and fails because the nodes already are added to a manual group. Is this a bug or is there are reason to do this?

apiVersion: v1
kind: Service
metadata:
  name: wikijs
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
  labels:
    app: wikijs
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 3000
  type: ClusterIP
  selector:
    app: wikijs

apiVersion: networking.gke.io/v1beta2
kind: ManagedCertificate
metadata:
  name: wikijs-domain-tld-2020-07-08
spec:
  domains:
    - wikijs.domain.tld

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: wikijs
  annotations:
    kubernetes.io/ingress.class: "gce"
    kubernetes.io/ingress.global-static-ip-name: "1.2.3.4"
    networking.gke.io/managed-certificates: "wikijs-2020-07-08"
spec:
  tls:
  - hosts:
    - wikijs.domain.tld
  rules:
  - host: wikijs.domain.tld
    http:
      paths:
      - path: /
        backend:
          serviceName: wikijs
          servicePort: 80

kubectl describe service wikijs

Name:              wikijs
Namespace:         default
Labels:            app=wikijs
Annotations:       cloud.google.com/neg: {"ingress": true}
                   cloud.google.com/neg-status:
                     {"network_endpoint_groups":{"80":"k8s1-f1d6835a-default-wikijs-80-f8349d79"},"zones":["europe-west1-b","europe-west1-c","europe-west1-d"]}
                   kubectl.kubernetes.io/last-applied-configuration:
                     {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"cloud.google.com/neg":"{\"ingress\": true}"},"labels":{"app":"wikijs"},"na...
Selector:          app=wikijs
Type:              ClusterIP
IP:                10.111.14.185
Port:              http  80/TCP
TargetPort:        3000/TCP
Endpoints:         
Session Affinity:  None
Events:            <none>

kubectl describe ingresses wikijs

Name:             wikijs
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (10.44.0.6:8080)
TLS:
  SNI routes wikijs.domain.tld
Rules:
  Host                     Path  Backends
  ----                     ----  --------
  wikijs.domain.tld  
                           /   wikijs:80 (10.44.4.5:3000)
Annotations:
  ...

  kubernetes.io/ingress.class:                  gce
  kubernetes.io/ingress.global-static-ip-name:  1.2.3.4
  networking.gke.io/managed-certificates:       wikijs-domain-tld-2020-07-08
  ingress.gcp.kubernetes.io/pre-shared-cert:    mcrt-21e827f1-82e7-4255-bd1d-3ebdc7ecfc8d
Events:
  Type     Reason  Age               From                     Message
  ----     ------  ----              ----                     -------
  Normal   ADD     2m47s             loadbalancer-controller  default/wikijs
  Warning  Sync    6s (x5 over 98s)  loadbalancer-controller  Error during sync: error running backend syncing routine: received errors when updating backend service: googleapi: Error 400: INSTANCE_IN_MULTIPLE_LOAD_BALANCED_IGS - Validation failed for instance 'projects/someproject/zones/europe-west1-b/instances/gke-public-default-pool-cdc4d452-ou7a': instance may belong to at most one load-balanced instance group.
googleapi: Error 400: INSTANCE_IN_MULTIPLE_LOAD_BALANCED_IGS - Validation failed for instance 'projects/someproject/zones/europe-west1-b/instances/gke-public-default-pool-cdc4d452-ou7a': instance may belong to at most one load-balanced instance group.

[rramkumar1]

Assigning @freehan to take a look

[bowei]

@rramkumar1 can you take this bug back from freehan?

[bowei]

@tlbdk can you post your cluster version

[tlbdk]

1.16.11-gke.5

[freehan]

The fix is already merged #1105
It is included in the 1.17+ versions.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.