Introduce Azure Event Grid as event destination

[tomkerkhove]

Proposal

Introduce Azure Event Grid as event destination:

apiVersion: events.keda.sh/v1alpha1
kind: EventSource # Or ClusterEventSource
metadata:
  name: operations-cross-cluster-events
spec:
  destination:
    azureEventgrid:
      topicEndpoint: https://{resource-name}.{region}.eventgrid.azure.net/api/events # Mandatory
      authentication: # End-users must use accessKey or activeDirectory
        accessKey:
          # Allow end-users to pull information from Kubernetes secret or from TriggerAuthentication resources
          valueFrom:
            secretKeyRef:
              name: secrets-operations-events
              key: webhook-api-key
            triggerAuthenticationRef:
              name: trigger-auth-sample
              parameterName: eventGridAuth
        activeDirectory:
          tenantId: xyz
          clientApplication:
            id: ABC
            secret:
              valueFrom:
                secretKeyRef:
                  name: secrets-operations-events
                  key: webhook-api-key
                triggerAuthenticationRef:
                  name: trigger-auth-sample
                  parameterName: eventGridAuth
          managedIdentity:
            valueFrom:
              triggerAuthenticationRef:
                name: trigger-auth-sample

Use-Case

Automatically forward CloudEvents to Azure Event Grid without HTTP endpoint in the middle.

Anything else?

No response

[tomkerkhove]

@kedacore/keda-maintainers Let me know what you think of the spec design but I'm open to suggestions

[tomkerkhove]

With regards to the e2e infrastructure for this, what we will need to add is: (tracked in kedacore/testing-infrastructure#144)

• An Azure Event Grid Topic to which we will push events
• An Azure Service Bus Namespace + Topic that is used to subscribe to events, from the above Azure Event Grid Topic.

For every test run + test, we need to:

1. Create a new subscription on the Service Bus Topic that will automatically delete in 1 day (ID: Random but should be logged)
2. Create a new subscription on the topic with respective filters (specific per test case) and propagate all tests to the Service Bus Topic
3. End-to-end test starts an async poller that gets all messages from subscription in 1)
4. Start test and trigger events that need to be emitted
5. Plow through received events from 3) in the test and make sure that the expected tests were processed

Here is an example of how I typically do this:

This is the only way that I've seen how you can securely and reliable do integration tests with Azure Event Grid.

[JorTurFer]

I'm a noobie with Event Grid, so I can't give you any feedback about how to test it, sorry
But I can support you with the infra changes if you need it