Skip to content
This repository has been archived by the owner on Oct 24, 2023. It is now read-only.

Kubediff find differences on missing defaults #84

Closed
nmiculinic opened this issue Feb 18, 2019 · 13 comments
Closed

Kubediff find differences on missing defaults #84

nmiculinic opened this issue Feb 18, 2019 · 13 comments

Comments

@nmiculinic
Copy link

kubediff should ignore the differences on missing defaults in the .yaml files. Poking through the source code you're getting the object (kubectl get), whilst instead should do something similar to kubectl diff
@nmiculinic
Copy link
Author

Here is what kubectl diff does for -v=9. Basically gets the object json and pathes with dry-run the file:

I0218 14:02:58.423434   46043 round_trippers.go:438] GET https://10.251.0.133:6443/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy 200 OK in 33 milliseconds
I0218 14:02:58.423526   46043 round_trippers.go:444] Response Headers:
I0218 14:02:58.423549   46043 round_trippers.go:447]     Content-Type: application/json
I0218 14:02:58.423588   46043 round_trippers.go:447]     Content-Length: 766
I0218 14:02:58.423663   46043 round_trippers.go:447]     Date: Mon, 18 Feb 2019 13:02:58 GMT
I0218 14:02:58.423729   46043 request.go:942] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"gitlab-deploy","namespace":"ascalia-dev","selfLink":"/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy","uid":"70371a68-2878-11e9-80f5-0242b121b722","resourceVersion":"466307","creationTimestamp":"2019-02-04T12:28:55Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"rbac.authorization.k8s.io/v1\",\"kind\":\"Role\",\"metadata\":{\"annotations\":{},\"name\":\"gitlab-deploy\",\"namespace\":\"ascalia-dev\"},\"rules\":[{\"apiGroups\":[\"extensions\"],\"resources\":[\"deployments\"],\"verbs\":[\"update\",\"get\"]}]}\n"}},"rules":[{"verbs":["update","get","patch"],"apiGroups":["extensions"],"resources":["deployments"]}]}
I0218 14:02:58.425157   46043 request.go:942] Request Body: {"metadata":{"resourceVersion":"466307"},"rules":[{"apiGroups":["extensions"],"resources":["deployments"],"verbs":["get","update","patch"]}]}
I0218 14:02:58.425218   46043 round_trippers.go:419] curl -k -v -XPATCH  -H "Accept: application/json" -H "Content-Type: application/strategic-merge-patch+json" -H "User-Agent: kubectl/v1.13.3 (darwin/amd64) kubernetes/721bfa7" 'https://10.251.0.133:6443/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy?dryRun=All'
I0218 14:02:58.463888   46043 round_trippers.go:438] PATCH https://10.251.0.133:6443/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy?dryRun=All 200 OK in 38 milliseconds
I0218 14:02:58.463933   46043 round_trippers.go:444] Response Headers:
I0218 14:02:58.463951   46043 round_trippers.go:447]     Content-Length: 766
I0218 14:02:58.463966   46043 round_trippers.go:447]     Date: Mon, 18 Feb 2019 13:02:58 GMT
I0218 14:02:58.463983   46043 round_trippers.go:447]     Content-Type: application/json
I0218 14:02:58.464045   46043 request.go:942] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"gitlab-deploy","namespace":"ascalia-dev","selfLink":"/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy","uid":"70371a68-2878-11e9-80f5-0242b121b722","resourceVersion":"466307","creationTimestamp":"2019-02-04T12:28:55Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"rbac.authorization.k8s.io/v1\",\"kind\":\"Role\",\"metadata\":{\"annotations\":{},\"name\":\"gitlab-deploy\",\"namespace\":\"ascalia-dev\"},\"rules\":[{\"apiGroups\":[\"extensions\"],\"resources\":[\"deployments\"],\"verbs\":[\"update\",\"get\"]}]}\n"}},"rules":[{"verbs":["get","update","patch"],"apiGroups":["extensions"],"resources":["deployments"]}]}

@nmiculinic
Copy link
Author

Related to #37

@nmiculinic
Copy link
Author

Or maybe this is a repeat of #37 , since all errors are those "missing" elements in the array. Anyhow using kubectl diff is much more stable and maintained solution anyhow, or even accessing the API server endpoints directly.

@samiyaakhtar
Copy link

+1, looking for a response to this as well.

@bboreham
Copy link
Contributor

Can you give an example of some input data that causes this?

@nmiculinic
Copy link
Author

Actually here you go:

 fn-k8s-ops (master) ✔ kubediff k8s-manifests/devtools/gitlab/charts/gitlab-runner/templates/deployment.yaml                                                                                                  k8s [devtools@fashionnetwork]
## devtools/gitlab-gitlab-runner (Deployment.v1beta1.extensions)

.spec.template.spec.containers: 'element [0]' missing
## devtools/gitlab-gitlab-runner (Deployment.v1beta1.extensions)

.spec.template.spec.initContainers: 'element [0]' missing
 fn-k8s-ops (master) ✔                                                                                                                                                                                        k8s [devtools@fashionnetwork]

And the yaml file:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-gitlab-runner
  labels:
    app: gitlab-gitlab-runner
    chart: gitlab-runner-0.3.0
    release: gitlab
    heritage: Tiller
  namespace: devtools
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: gitlab-gitlab-runner
      annotations:
        checksum/configmap: dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7
        checksum/secrets: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
        prometheus.io/scrape: "true"
        prometheus.io/port: "9252"
    spec:
      securityContext:
        runAsUser: 100
        fsGroup: 65533
      initContainers:
      - name: configure
        command:
        - sh
        - /config/configure
        image: gitlab/gitlab-runner:alpine-v11.9.0
        env:
        - name: CI_SERVER_URL
          value: https://gitlab.fashionnetwork.com
        - name: CLONE_URL
          value: ""
        - name: RUNNER_EXECUTOR
          value: kubernetes
        - name: REGISTER_LOCKED
          value: "false"
        - name: RUNNER_TAG_LIST
          value: docker
        - name: KUBERNETES_IMAGE
          value: ubuntu:16.04
        - name: KUBERNETES_PRIVILEGED
          value: "true"
        - name: KUBERNETES_NAMESPACE
          value: gitlab-jobs
        - name: KUBERNETES_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_SERVICE_ACCOUNT
          value: ""
        - name: KUBERNETES_SERVICE_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_SERVICE_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_SERVICE_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_SERVICE_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_HELPER_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_HELPER_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_HELPER_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_IMAGE
          value: ""
        - name: KUBERNETES_PULL_POLICY
          value: always
        - name: CACHE_TYPE
          value: s3
        - name: CACHE_PATH
          value: gitlab_runner
        - name: CACHE_SHARED
          value: "true"
        - name: CACHE_S3_SERVER_ADDRESS
          value: s3.amazonaws.com
        - name: CACHE_S3_BUCKET_NAME
          value: gitlab-fn-runner-cache
        - name: CACHE_S3_BUCKET_LOCATION
          value: eu-central-1
        volumeMounts:
        - name: runner-secrets
          mountPath: /secrets
          readOnly: false
        - name: scripts
          mountPath: /config
          readOnly: true
        - name: init-runner-secrets
          mountPath: /init-secrets
          readOnly: true
        resources: {}
      serviceAccountName: gitlab-gitlab-runner
      containers:
      - name: gitlab-gitlab-runner
        image: gitlab/gitlab-runner:alpine-v11.9.0
        imagePullPolicy: IfNotPresent
        command:
        - /bin/bash
        - /scripts/entrypoint
        env:
        - name: CI_SERVER_URL
          value: https://gitlab.fashionnetwork.com
        - name: CLONE_URL
          value: ""
        - name: RUNNER_EXECUTOR
          value: kubernetes
        - name: REGISTER_LOCKED
          value: "false"
        - name: RUNNER_TAG_LIST
          value: docker
        - name: KUBERNETES_IMAGE
          value: ubuntu:16.04
        - name: KUBERNETES_PRIVILEGED
          value: "true"
        - name: KUBERNETES_NAMESPACE
          value: gitlab-jobs
        - name: KUBERNETES_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_SERVICE_ACCOUNT
          value: ""
        - name: KUBERNETES_SERVICE_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_SERVICE_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_SERVICE_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_SERVICE_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_HELPER_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_HELPER_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_HELPER_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_IMAGE
          value: ""
        - name: KUBERNETES_PULL_POLICY
          value: always
        - name: CACHE_TYPE
          value: s3
        - name: CACHE_PATH
          value: gitlab_runner
        - name: CACHE_SHARED
          value: "true"
        - name: CACHE_S3_SERVER_ADDRESS
          value: s3.amazonaws.com
        - name: CACHE_S3_BUCKET_NAME
          value: gitlab-fn-runner-cache
        - name: CACHE_S3_BUCKET_LOCATION
          value: eu-central-1
        livenessProbe:
          exec:
            command:
            - /bin/bash
            - /scripts/check-live
          initialDelaySeconds: 60
          timeoutSeconds: 1
          periodSeconds: 10
          successThreshold: 1
          failureThreshold: 3
        readinessProbe:
          exec:
            command:
            - /usr/bin/pgrep
            - gitlab.*runner
          initialDelaySeconds: 10
          timeoutSeconds: 1
          periodSeconds: 10
          successThreshold: 1
          failureThreshold: 3
        ports:
        - name: metrics
          containerPort: 9252
        volumeMounts:
        - name: runner-secrets
          mountPath: /secrets
        - name: etc-gitlab-runner
          mountPath: /home/gitlab-runner/.gitlab-runner
        - name: scripts
          mountPath: /scripts
        resources: {}
      volumes:
      - name: runner-secrets
        emptyDir:
          medium: Memory
      - name: etc-gitlab-runner
        emptyDir:
          medium: Memory
      - name: init-runner-secrets
        projected:
          sources:
          - secret:
              name: gitlab-aws-s3
          - secret:
              name: gitlab-gitlab-runner-secret
              items:
              - key: runner-registration-token
                path: runner-registration-token
              - key: runner-token
                path: runner-token
      - name: scripts
        configMap:
          name: gitlab-gitlab-runner

@bboreham
Copy link
Contributor

Can you post what kubectl get -o yaml returns for that object so I can see the comparison, please.

@bboreham
Copy link
Contributor

Within a dictionary, kubediff only compares items that are present in the source, i.e. in the file it is given. In this way, most items defaulted by Kubernetes are ignored.

@nmiculinic
Copy link
Author 

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "11"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"gitlab-gitlab-runner","chart":"gitlab-runner-0.3.0","heritage":"Tiller","release":"gitlab"},"name":"gitlab-gitlab-runner","namespace":"devtools"},"spec":{"replicas":1,"template":{"metadata":{"annotations":{"checksum/configmap":"dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7","checksum/secrets":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","prometheus.io/port":"9252","prometheus.io/scrape":"true"},"labels":{"app":"gitlab-gitlab-runner"}},"spec":{"containers":[{"command":["/bin/bash","/scripts/entrypoint"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["/bin/bash","/scripts/check-live"]},"failureThreshold":3,"initialDelaySeconds":60,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"name":"gitlab-gitlab-runner","ports":[{"containerPort":9252,"name":"metrics"}],"readinessProbe":{"exec":{"command":["/usr/bin/pgrep","gitlab.*runner"]},"failureThreshold":3,"initialDelaySeconds":10,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets"},{"mountPath":"/home/gitlab-runner/.gitlab-runner","name":"etc-gitlab-runner"},{"mountPath":"/scripts","name":"scripts"}]}],"initContainers":[{"command":["sh","/config/configure"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","name":"configure","resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets","readOnly":false},{"mountPath":"/config","name":"scripts","readOnly":true},{"mountPath":"/init-secrets","name":"init-runner-secrets","readOnly":true}]}],"securityContext":{"fsGroup":65533,"runAsUser":100},"serviceAccountName":"gitlab-gitlab-runner","volumes":[{"emptyDir":{"medium":"Memory"},"name":"runner-secrets"},{"emptyDir":{"medium":"Memory"},"name":"etc-gitlab-runner"},{"name":"init-runner-secrets","projected":{"sources":[{"secret":{"name":"gitlab-aws-s3"}},{"secret":{"items":[{"key":"runner-registration-token","path":"runner-registration-token"},{"key":"runner-token","path":"runner-token"}],"name":"gitlab-gitlab-runner-secret"}}]}},{"configMap":{"name":"gitlab-gitlab-runner"},"name":"scripts"}]}}}}
  creationTimestamp: "2019-02-20T10:03:29Z"
  generation: 11
  labels:
    app: gitlab-gitlab-runner
    chart: gitlab-runner-0.3.0
    heritage: Tiller
    release: gitlab
  name: gitlab-gitlab-runner
  namespace: devtools
  resourceVersion: "8914046"
  selfLink: /apis/extensions/v1beta1/namespaces/devtools/deployments/gitlab-gitlab-runner
  uid: c5f662ea-34f6-11e9-95c5-061409733802
spec:
  progressDeadlineSeconds: 2147483647
  replicas: 1
  revisionHistoryLimit: 2147483647
  selector:
    matchLabels:
      app: gitlab-gitlab-runner
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      annotations:
        checksum/configmap: dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7
        checksum/secrets: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
        prometheus.io/port: "9252"
        prometheus.io/scrape: "true"
      creationTimestamp: null
      labels:
        app: gitlab-gitlab-runner
    spec:
      containers:
      - command:
        - /bin/bash
        - /scripts/entrypoint
        env:
        - name: CI_SERVER_URL
          value: https://gitlab.fashionnetwork.com
        - name: CLONE_URL
        - name: RUNNER_EXECUTOR
          value: kubernetes
        - name: REGISTER_LOCKED
          value: "false"
        - name: RUNNER_TAG_LIST
          value: docker
        - name: KUBERNETES_IMAGE
          value: ubuntu:16.04
        - name: KUBERNETES_PRIVILEGED
          value: "true"
        - name: KUBERNETES_NAMESPACE
          value: gitlab-jobs
        - name: KUBERNETES_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_SERVICE_ACCOUNT
        - name: KUBERNETES_SERVICE_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_SERVICE_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_SERVICE_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_SERVICE_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_HELPER_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_HELPER_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_HELPER_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_IMAGE
        - name: KUBERNETES_PULL_POLICY
          value: always
        - name: CACHE_TYPE
          value: s3
        - name: CACHE_PATH
          value: gitlab_runner
        - name: CACHE_SHARED
          value: "true"
        - name: CACHE_S3_SERVER_ADDRESS
          value: s3.amazonaws.com
        - name: CACHE_S3_BUCKET_NAME
          value: gitlab-fn-runner-cache
        - name: CACHE_S3_BUCKET_LOCATION
          value: eu-central-1
        image: gitlab/gitlab-runner:alpine-v11.9.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          exec:
            command:
            - /bin/bash
            - /scripts/check-live
          failureThreshold: 3
          initialDelaySeconds: 60
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: gitlab-gitlab-runner
        ports:
        - containerPort: 9252
          name: metrics
          protocol: TCP
        readinessProbe:
          exec:
            command:
            - /usr/bin/pgrep
            - gitlab.*runner
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /secrets
          name: runner-secrets
        - mountPath: /home/gitlab-runner/.gitlab-runner
          name: etc-gitlab-runner
        - mountPath: /scripts
          name: scripts
      dnsPolicy: ClusterFirst
      initContainers:
      - command:
        - sh
        - /config/configure
        env:
        - name: CI_SERVER_URL
          value: https://gitlab.fashionnetwork.com
        - name: CLONE_URL
        - name: RUNNER_EXECUTOR
          value: kubernetes
        - name: REGISTER_LOCKED
          value: "false"
        - name: RUNNER_TAG_LIST
          value: docker
        - name: KUBERNETES_IMAGE
          value: ubuntu:16.04
        - name: KUBERNETES_PRIVILEGED
          value: "true"
        - name: KUBERNETES_NAMESPACE
          value: gitlab-jobs
        - name: KUBERNETES_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_SERVICE_ACCOUNT
        - name: KUBERNETES_SERVICE_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_SERVICE_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_SERVICE_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_SERVICE_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_HELPER_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_HELPER_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_HELPER_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_IMAGE
        - name: KUBERNETES_PULL_POLICY
          value: always
        - name: CACHE_TYPE
          value: s3
        - name: CACHE_PATH
          value: gitlab_runner
        - name: CACHE_SHARED
          value: "true"
        - name: CACHE_S3_SERVER_ADDRESS
          value: s3.amazonaws.com
        - name: CACHE_S3_BUCKET_NAME
          value: gitlab-fn-runner-cache
        - name: CACHE_S3_BUCKET_LOCATION
          value: eu-central-1
        image: gitlab/gitlab-runner:alpine-v11.9.0
        imagePullPolicy: IfNotPresent
        name: configure
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /secrets
          name: runner-secrets
        - mountPath: /config
          name: scripts
          readOnly: true
        - mountPath: /init-secrets
          name: init-runner-secrets
          readOnly: true
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 65533
        runAsUser: 100
      serviceAccount: gitlab-gitlab-runner
      serviceAccountName: gitlab-gitlab-runner
      terminationGracePeriodSeconds: 30
      volumes:
      - emptyDir:
          medium: Memory
        name: runner-secrets
      - emptyDir:
          medium: Memory
        name: etc-gitlab-runner
      - name: init-runner-secrets
        projected:
          defaultMode: 420
          sources:
          - secret:
              name: gitlab-aws-s3
          - secret:
              items:
              - key: runner-registration-token
                path: runner-registration-token
              - key: runner-token
                path: runner-token
              name: gitlab-gitlab-runner-secret
      - configMap:
          defaultMode: 420
          name: gitlab-gitlab-runner
        name: scripts
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2019-02-20T10:03:29Z"
    lastUpdateTime: "2019-02-20T10:03:29Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 11
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "11"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"gitlab-gitlab-runner","chart":"gitlab-runner-0.3.0","heritage":"Tiller","release":"gitlab"},"name":"gitlab-gitlab-runner","namespace":"devtools"},"spec":{"replicas":1,"template":{"metadata":{"annotations":{"checksum/configmap":"dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7","checksum/secrets":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","prometheus.io/port":"9252","prometheus.io/scrape":"true"},"labels":{"app":"gitlab-gitlab-runner"}},"spec":{"containers":[{"command":["/bin/bash","/scripts/entrypoint"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["/bin/bash","/scripts/check-live"]},"failureThreshold":3,"initialDelaySeconds":60,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"name":"gitlab-gitlab-runner","ports":[{"containerPort":9252,"name":"metrics"}],"readinessProbe":{"exec":{"command":["/usr/bin/pgrep","gitlab.*runner"]},"failureThreshold":3,"initialDelaySeconds":10,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets"},{"mountPath":"/home/gitlab-runner/.gitlab-runner","name":"etc-gitlab-runner"},{"mountPath":"/scripts","name":"scripts"}]}],"initContainers":[{"command":["sh","/config/configure"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","name":"configure","resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets","readOnly":false},{"mountPath":"/config","name":"scripts","readOnly":true},{"mountPath":"/init-secrets","name":"init-runner-secrets","readOnly":true}]}],"securityContext":{"fsGroup":65533,"runAsUser":100},"serviceAccountName":"gitlab-gitlab-runner","volumes":[{"emptyDir":{"medium":"Memory"},"name":"runner-secrets"},{"emptyDir":{"medium":"Memory"},"name":"etc-gitlab-runner"},{"name":"init-runner-secrets","projected":{"sources":[{"secret":{"name":"gitlab-aws-s3"}},{"secret":{"items":[{"key":"runner-registration-token","path":"runner-registration-token"},{"key":"runner-token","path":"runner-token"}],"name":"gitlab-gitlab-runner-secret"}}]}},{"configMap":{"name":"gitlab-gitlab-runner"},"name":"scripts"}]}}}}
  creationTimestamp: "2019-02-20T10:03:29Z"
  generation: 11
  labels:
    app: gitlab-gitlab-runner
    chart: gitlab-runner-0.3.0
    heritage: Tiller
    release: gitlab
  name: gitlab-gitlab-runner
  namespace: devtools
  resourceVersion: "8914046"
  selfLink: /apis/extensions/v1beta1/namespaces/devtools/deployments/gitlab-gitlab-runner
  uid: c5f662ea-34f6-11e9-95c5-061409733802
spec:
  progressDeadlineSeconds: 2147483647
  replicas: 1
  revisionHistoryLimit: 2147483647
  selector:
    matchLabels:
      app: gitlab-gitlab-runner
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      annotations:
        checksum/configmap: dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7
        checksum/secrets: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
        prometheus.io/port: "9252"
        prometheus.io/scrape: "true"
      creationTimestamp: null
      labels:
        app: gitlab-gitlab-runner
    spec:
      containers:
      - command:
        - /bin/bash
        - /scripts/entrypoint
        env:
        - name: CI_SERVER_URL
          value: https://gitlab.fashionnetwork.com
        - name: CLONE_URL
        - name: RUNNER_EXECUTOR
          value: kubernetes
        - name: REGISTER_LOCKED
          value: "false"
        - name: RUNNER_TAG_LIST
          value: docker
        - name: KUBERNETES_IMAGE
          value: ubuntu:16.04
        - name: KUBERNETES_PRIVILEGED
          value: "true"
        - name: KUBERNETES_NAMESPACE
          value: gitlab-jobs
        - name: KUBERNETES_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_SERVICE_ACCOUNT
        - name: KUBERNETES_SERVICE_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_SERVICE_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_SERVICE_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_SERVICE_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_HELPER_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_HELPER_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_HELPER_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_IMAGE
        - name: KUBERNETES_PULL_POLICY
          value: always
        - name: CACHE_TYPE
          value: s3
        - name: CACHE_PATH
          value: gitlab_runner
        - name: CACHE_SHARED
          value: "true"
        - name: CACHE_S3_SERVER_ADDRESS
          value: s3.amazonaws.com
        - name: CACHE_S3_BUCKET_NAME
          value: gitlab-fn-runner-cache
        - name: CACHE_S3_BUCKET_LOCATION
          value: eu-central-1
        image: gitlab/gitlab-runner:alpine-v11.9.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          exec:
            command:
            - /bin/bash
            - /scripts/check-live
          failureThreshold: 3
          initialDelaySeconds: 60
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: gitlab-gitlab-runner
        ports:
        - containerPort: 9252
          name: metrics
          protocol: TCP
        readinessProbe:
          exec:
            command:
            - /usr/bin/pgrep
            - gitlab.*runner
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /secrets
          name: runner-secrets
        - mountPath: /home/gitlab-runner/.gitlab-runner
          name: etc-gitlab-runner
        - mountPath: /scripts
          name: scripts
      dnsPolicy: ClusterFirst
      initContainers:
      - command:
        - sh
        - /config/configure
        env:
        - name: CI_SERVER_URL
          value: https://gitlab.fashionnetwork.com
        - name: CLONE_URL
        - name: RUNNER_EXECUTOR
          value: kubernetes
        - name: REGISTER_LOCKED
          value: "false"
        - name: RUNNER_TAG_LIST
          value: docker
        - name: KUBERNETES_IMAGE
          value: ubuntu:16.04
        - name: KUBERNETES_PRIVILEGED
          value: "true"
        - name: KUBERNETES_NAMESPACE
          value: gitlab-jobs
        - name: KUBERNETES_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_SERVICE_ACCOUNT
        - name: KUBERNETES_SERVICE_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_SERVICE_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_SERVICE_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_SERVICE_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_CPU_LIMIT
          value: 200m
        - name: KUBERNETES_HELPER_MEMORY_LIMIT
          value: 256Mi
        - name: KUBERNETES_HELPER_CPU_REQUEST
          value: 100m
        - name: KUBERNETES_HELPER_MEMORY_REQUEST
          value: 128Mi
        - name: KUBERNETES_HELPER_IMAGE
        - name: KUBERNETES_PULL_POLICY
          value: always
        - name: CACHE_TYPE
          value: s3
        - name: CACHE_PATH
          value: gitlab_runner
        - name: CACHE_SHARED
          value: "true"
        - name: CACHE_S3_SERVER_ADDRESS
          value: s3.amazonaws.com
        - name: CACHE_S3_BUCKET_NAME
          value: gitlab-fn-runner-cache
        - name: CACHE_S3_BUCKET_LOCATION
          value: eu-central-1
        image: gitlab/gitlab-runner:alpine-v11.9.0
        imagePullPolicy: IfNotPresent
        name: configure
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /secrets
          name: runner-secrets
        - mountPath: /config
          name: scripts
          readOnly: true
        - mountPath: /init-secrets
          name: init-runner-secrets
          readOnly: true
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 65533
        runAsUser: 100
      serviceAccount: gitlab-gitlab-runner
      serviceAccountName: gitlab-gitlab-runner
      terminationGracePeriodSeconds: 30
      volumes:
      - emptyDir:
          medium: Memory
        name: runner-secrets
      - emptyDir:
          medium: Memory
        name: etc-gitlab-runner
      - name: init-runner-secrets
        projected:
          defaultMode: 420
          sources:
          - secret:
              name: gitlab-aws-s3
          - secret:
              items:
              - key: runner-registration-token
                path: runner-registration-token
              - key: runner-token
                path: runner-token
              name: gitlab-gitlab-runner-secret
      - configMap:
          defaultMode: 420
          name: gitlab-gitlab-runner
        name: scripts
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2019-02-20T10:03:29Z"
    lastUpdateTime: "2019-02-20T10:03:29Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 11
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

@bboreham
Copy link
Contributor

The one thing I can see is that empty environment variables, e.g.

        - name: CLONE_URL
          value: ""

come back with a missing value which causes a diff.

What made you think it was defaults?

@nmiculinic
Copy link
Author

IIRC I think I observed similar behavior when timeoutSeconds wasn't specified in the liveness/rediness probe. (( default being 1s IIRC ))

@bboreham
Copy link
Contributor

bboreham commented Apr 16, 2019 •
edited
Loading

If you have an example, please post. Within a dictionary such as .spec.template.spec.containers[0].readinessProbe, kubediff only compares items that are present in the source.

@bboreham
Copy link
Contributor

Closing - if you get some concrete evidence please post and re-open.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@samiyaakhtar @nmiculinic @bboreham