Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add and enable aws for fluent bit health check #975

Merged

Conversation

PettitWesley
Copy link
Contributor

Description of changes

Copied from here with permission: https://github.com/aws/eks-charts/pull/947/files

Checklist

  • Added/modified documentation as required (such as the README.md for modified charts)
  • Incremented the chart version in Chart.yaml for the modified chart(s)
  • Manually tested. Describe what testing was done in the testing section below
  • Make sure the title of the PR is a good description that can go into the release notes

Testing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@PettitWesley
Copy link
Contributor Author

Testing info:

---
# Source: aws-for-fluent-bit/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: aws-for-fluent-bit
  namespace: default
  labels:
    helm.sh/chart: aws-for-fluent-bit-0.1.28
    app.kubernetes.io/name: aws-for-fluent-bit
    app.kubernetes.io/instance: aws-for-fluent-bit
    app.kubernetes.io/version: "2.31.11"
    app.kubernetes.io/managed-by: Helm
---
# Source: aws-for-fluent-bit/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-for-fluent-bit
  namespace: default
  labels:
    helm.sh/chart: aws-for-fluent-bit-0.1.28
    app.kubernetes.io/name: aws-for-fluent-bit
    app.kubernetes.io/instance: aws-for-fluent-bit
    app.kubernetes.io/version: "2.31.11"
    app.kubernetes.io/managed-by: Helm
data:
  fluent-bit.conf: |
    [SERVICE]
        HTTP_Server  On
        HTTP_Listen  0.0.0.0
        HTTP_PORT    2020
        Health_Check On 
        HC_Errors_Count 5 
        HC_Retry_Failure_Count 5 
        HC_Period 5 
        
        Parsers_File /fluent-bit/parsers/parsers.conf
    [INPUT]
        Name              tail
        Tag               kube.*
        Path              /var/log/containers/*.log
        DB                /var/log/flb_kube.db
        Parser            docker
        Docker_Mode       On
        Mem_Buf_Limit     5MB
        Skip_Long_Lines   On
        Refresh_Interval  10
    [FILTER]
        Name                kubernetes
        Match               kube.*
        Kube_URL            https://kubernetes.default.svc.cluster.local:443
        Merge_Log           On
        Merge_Log_Key       data
        Keep_Log            On
        K8S-Logging.Parser  On
        K8S-Logging.Exclude On
        Buffer_Size         32k
    [OUTPUT]
        Name                  cloudwatch_logs
        Match                 *
        region                us-east-1
        log_group_name        /aws/eks/fluentbit-cloudwatch/logs
        log_group_template    /aws/eks/fluentbit-cloudwatch/workload/$kubernetes['namespace_name']
        log_stream_prefix     fluentbit-
        log_stream_template   $kubernetes['pod_name'].$kubernetes['container_name']
        auto_create_group     true
---
# Source: aws-for-fluent-bit/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: aws-for-fluent-bit
rules:
  - apiGroups: [""]
    resources:
      - namespaces
      - pods
      - pods/logs
      - nodes
      - nodes/proxy
    verbs: ["get", "list", "watch"]
  - apiGroups: ["policy"]
    resources: ["podsecuritypolicies"]
    verbs: ["use"]
    resourceNames:
      - aws-for-fluent-bit
---
# Source: aws-for-fluent-bit/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: aws-for-fluent-bit
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: aws-for-fluent-bit
subjects:
  - kind: ServiceAccount
    name: aws-for-fluent-bit
    namespace: default
---
# Source: aws-for-fluent-bit/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    helm.sh/chart: aws-for-fluent-bit-0.1.28
    app.kubernetes.io/name: aws-for-fluent-bit
    app.kubernetes.io/instance: aws-for-fluent-bit
    app.kubernetes.io/version: "2.31.11"
    app.kubernetes.io/managed-by: Helm
  name: aws-for-fluent-bit
  namespace: default
spec:
  ports:
  - name: monitor-agent
    port: 2020
    protocol: TCP
    targetPort: 2020
  selector:
    app.kubernetes.io/name: aws-for-fluent-bit
    app.kubernetes.io/instance: aws-for-fluent-bit
  sessionAffinity: None
  type: ClusterIP
---
# Source: aws-for-fluent-bit/templates/daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: aws-for-fluent-bit
  namespace: default
  labels:
    helm.sh/chart: aws-for-fluent-bit-0.1.28
    app.kubernetes.io/name: aws-for-fluent-bit
    app.kubernetes.io/instance: aws-for-fluent-bit
    app.kubernetes.io/version: "2.31.11"
    app.kubernetes.io/managed-by: Helm
spec:
  updateStrategy:
    type: RollingUpdate
  selector:
    matchLabels:
      app.kubernetes.io/name: aws-for-fluent-bit
      app.kubernetes.io/instance: aws-for-fluent-bit
  template:
    metadata:
      annotations:
        checksum/config: f02490ce05e9b48446db5d39b9480a0a5e713c8ee719bf5b6799d5e9a00aacb4
      labels:
        app.kubernetes.io/name: aws-for-fluent-bit
        app.kubernetes.io/instance: aws-for-fluent-bit
    spec:
      serviceAccountName: aws-for-fluent-bit
      dnsPolicy: ClusterFirst
      containers:
        - name: aws-for-fluent-bit
          imagePullPolicy: IfNotPresent
          image: "public.ecr.aws/aws-observability/aws-for-fluent-bit:2.31.11"
          volumeMounts:
            - name: fluentbit-config
              mountPath: /fluent-bit/etc/
            - mountPath: /var/log
              name: varlog
            - mountPath: /var/lib/docker/containers
              name: varlibdockercontainers
              readOnly: true
          resources:
            limits:
              memory: 250Mi
            requests:
              cpu: 50m
              memory: 50Mi
          livenessProbe:
            failureThreshold: 2
            httpGet:
              path: /api/v1/health
              port: 2020
              scheme: HTTP
            initialDelaySeconds: 30
            timeoutSeconds: 10
      volumes:
        - name: fluentbit-config
          configMap:
            name: aws-for-fluent-bit
        - hostPath:
            path: /var/log
          name: varlog
        - hostPath:
            path: /var/lib/docker/containers
          name: varlibdockercontainers

@PettitWesley PettitWesley merged commit 25dc508 into aws:master Jul 26, 2023
1 check passed
@PettitWesley PettitWesley deleted the add-aws-for-fluent-bit-healthchecks branch July 26, 2023 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants