Add SHA512 converter.

[mashhurs]

Description:

Adds a converter to generate SHA-512 digest. See the #34007 issue for more details.

Link to tracking Issue: #34007

Testing:

• Unit testing
• E2E
• Local testing
  • after applying changes, build opentelemetry-collector-contrib: cd opentelemetry-collector-contrib && make otelcontribcol
  • create config folder and place following config under otel-transform-sha512.yaml file:

    extensions:
      zpages:
        endpoint: 127.0.0.1:55679
    
    receivers:
      otlp:
        protocols:
          http:
            endpoint: 127.0.0.1:8080
    
    processors:
      batch:
      transform:
        log_statements:
          - context: log
            statements:
              - set(attributes["string.attribute"], SHA512(attributes["string.attribute"]))
    
    exporters:
      debug:
        verbosity: detailed
      elasticsearch:
        hosts: ["localhost:9200"]
        user: "elastic"
        password: "changeme"
        index: "otel-collector-sha256"
    
    service:
      telemetry:
        logs:
          level: debug
      pipelines:
        logs:
          receivers: [otlp]
          processors: [batch, transform]
          exporters: [elasticsearch, debug]
    
      extensions: [zpages]
    

  • run otel collector: bin/otelcontribcol_darwin_amd64 --config=config/otel-transform-sha512.yaml
  • send data

       curl --location '127.0.0.1:8080/v1/logs' \
    --header 'Content-Type: application/json' \
    --data '{
     "resourceLogs": [
       {
         "resource": {
           "attributes": [
             {
               "key": "service.name",
               "value": {
                 "stringValue": "my.service"
               }
             }
           ]
         },
         "scopeLogs": [
           {
             "scope": {
               "name": "my.library",
               "version": "1.0.0",
               "attributes": [
                 {
                   "key": "my.scope.attribute",
                   "value": {
                     "stringValue": "some scope attribute"
                   }
                 }
               ]
             },
             "logRecords": [
               {
                 "timeUnixNano": "1544712660300000000",
                 "observedTimeUnixNano": "1544712660300000000",
                 "severityNumber": 10,
                 "severityText": "Information",
                 "traceId": "5B8EFFF798038103D269B633813FC60C",
                 "spanId": "EEE19B7EC3C1B174",
                 "body": {
                   "stringValue": "Example log record"
                 },
                 "attributes": [
                   {
                     "key": "string.attribute",
                     "value": {
                       "stringValue": "sending some data for SHA512"
                     }
                   },
                   {
                     "key": "boolean.attribute",
                     "value": {
                       "boolValue": true
                     }
                   },
                   {
                     "key": "int.attribute",
                     "value": {
                       "intValue": "10"
                     }
                   },
                   {
                     "key": "double.attribute",
                     "value": {
                       "doubleValue": 637.704
                     }
                   },
                   {
                     "key": "array.attribute",
                     "value": {
                       "arrayValue": {
                         "values": [
                           {
                             "stringValue": "many"
                           },
                           {
                             "stringValue": "values"
                           }
                         ]
                       }
                     }
                   },
                   {
                     "key": "map.attribute",
                     "value": {
                       "kvlistValue": {
                         "values": [
                           {
                             "key": "some.map.key",
                             "value": {
                               "stringValue": "some value"
                             }
                           }
                         ]
                       }
                     }
                   }
                 ]
               }
             ]
           }
         ]
       }
     ]
    }'
    

  • you will be able to see SHA512 value on console

    ObservedTimestamp: 2018-12-13 14:51:00.3 +0000 UTC
    Timestamp: 2018-12-13 14:51:00.3 +0000 UTC
    SeverityText: Information
    SeverityNumber: Info2(10)
    Body: Str(Example log record)
    Attributes:
         -> string.attribute: Str(6cbe75a224033d0f145fd67e02cab35ce42a10cd1aa7b7030dd74e31af97037f2020d063305bc2552d1b9e43ca15872ea248c5b236df98085c690932a7f0ea49)
         -> boolean.attribute: Bool(true)
         -> int.attribute: Int(10)
         -> double.attribute: Double(637.704)
         -> array.attribute: Slice(["many","values"])
         -> map.attribute: Map({"some.map.key":"some value"})
    Trace ID: 5b8efff798038103d269b633813fc60c
    Span ID: eee19b7ec3c1b174
    

Documentation:

• Added to OTTL readme and changelogs generated. Let me please know if I am missing any doc references.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: mashhurs / name: Mashhur (12cdd5d, d42a47d, 954e821)

[atoulme]

Looks good, kicking off CI

[mashhurs]

Hm.., not sure why 'exporter/elasticsearchexporter/integrationtest' as part of group 'exporter-1' is failing. Current PR doesn't touch elasticsearch exporter (or relative) logic. Based on the comment in the test, it seems it is due to #30792 intermittent case.
Recent CI seems good, updating branch with main.

It seems to me, we need to disable the PQ to get a consistent CI status till the #30792 gets resolved.

sending_queue:
  enabled: true

[mashhurs]

Wooh, CI is 🟢 now, thank you @atoulme! Can you please continue your review @atoulme? Thank you!

[evan-bradley]

Looks good to me, thanks @mashhurs.

[evan-bradley]

@TylerHelmuth Please take a look.

[atoulme]

There is a trivial conflict to resolve on the README.md file, please take a look