Adding an all_available_flags behavior to providers and clients

[maxveldink]

This is carried over from a thread in the OF Slack channel.

One use case our team has had to patch into our OF client (and provider implementations) is the ability to see all available flags for a given evaluation context. This is a feature available in LaunchDarkly (our primary provider) and a few other providers as well.

Note. This is different from bulk_flag_evaluation (as discussed here). Our code needs to know the flag keys to evaluate this particular use case beforehand. We want to query an OF client (and, by proxy, the provider) to see all available flag keys for a given context.

There are two motivating uses for this:

1. We are building debug screens that allow for flipping certain features on and off. Ideally, the client programmatically built these screens without introducing specific flag evaluation points.
2. We're hydrating a GQL response on the server with all available flags so our mobile clients can do a faster first paint without having to wait for any decision from flag evaluation

Our current patch adds two new methods to the provider and client interfaces. This seems too large of a change to support a use case that not all providers support.

One idea was to include this as optional behavior on the provider metadata object. We don't store much here right now, and it feels different enough than flag evaluation that adding a new, required method implementation for all providers seems overly restrictive. We'd then be able to access provider metadata through the client's metadata (which I don't believe is currently a part of the spec).

Further, maybe this is an opportunity to add a more general, optional key-value storage to provider metadata. We store some non-standard information in our in-house providers, but we might want to shore those up more in the spec.

I'm curious if others have a similar need for this behavior, how they've approached it, and if we're open to adding something to the specification to standardize this concept. 🙏🏻

[weyert]

I think this is a good idea. I can imagine that for some flag providers/servie when you want to querying all available flags you need to provide a different token?

I assume you want this on both the server and web OpenFeature clients?

Also I do wonder if we want to care and/or warn about potential cost implication of using these commands. Typically, bulk evaluation costs a lot more than a single flag evaluation in providers. I can imagine that for some providers when querying all available flags the same applies.

[austindrenski]

Also I do wonder if we want to care and/or warn about potential cost implication of using these commands. Typically, bulk evaluation costs a lot more than a single flag evaluation in providers. I can imagine that for some providers when querying all available flags the same applies.

Just my two cents, but this sounds like a provider responsibility more so than the SDK (e.g. flagd docs should contain the warning, not the OpenFeature itself nor its language-specific SDKs).

Example from the OpenTelemetry Collector's awscloudwatchmetricsreceiver:

AWS Costs

This receiver uses the GetMetricData API call, this call is not in the AWS free tier. Please refer to Amazon's pricing for further information about expected costs.

[maxveldink]

Yeah, I had considered the performance cost of this, but I largely agree with @austindrenski that this could probably be clarified in the provider docs itself. Same with any additional credentialing that probably needs to be passed during provider initialization (and up to the provider to document) if the application intends to use the all flags behavior.

| I assume you want this on both the server and web OpenFeature clients?

Yep! I believe this could be reused for both. We've had internal discussions about leaking flags and if that matters to our use case. For us, the client SDKs are connected to specific projects and are defaulted to public knowledge, so we don't see a large issue with that.

[weyert]

Yeah, that's a problem. Typically there is a high chance that you leak backend flags (e.g. ops flags) to the frontend when you don't have separate projects. At least that's the case when using PostHog

[beeme1mr]

LaunchDarkly and others allow you to specify if a feature flag should be exposed to client-side SDKs.

[beeme1mr]

Hey @maxveldink, thanks for starting the discussion. The use case makes sense and would likely be straightforward to implement, at least on client-side SDKs. Extending provider metadata may be a good option, but we could do this in a couple of ways.

How should the structure of the all flags object look? LaunchDarkly returns a map of flag keys and the values based on the current context. We could do something similar or possibly a list of evaluation details.

[maxveldink]

List of evaluation details makes sense to me and would give us all the information we needed. We might want to make just the flag_keys available, but generally having the evaluation as well would be very helpful.

[weyert]

My experience with the flag providers I used is that you would get the flag definition back (including the evaluation rules/filters) and require a non-public token. You can't get all these details without a such token that allows also managing the tokens. But I will double check :)

[beeme1mr]

@weyert, there are a few providers that send the ruleset to the client but it's fairly uncommon. The evaluation details don't only require the flag value but can optionally include other metadata like variant, reason, etc.

[konradjniemiec]

I feel like this is tricky to define without scoping to backend or frontend SDKs

although if we don't return any information about the keys just the list, the client can then request whatever information makes the most sense, maybe a bulk evaluation for example, since these two things seem to go hand in hand

[beeme1mr]

I think a list of flag keys would be the safest option to start with.

[amahiwa-jp]

I agree, we have a use case where we want to use OpenFeature to have a single interface for multiple in-house feature flag providers and consolidate them on a single UI. Right now, we cannot fully use OpenFeature since there is no support for a getAllFlags operation or something similar.

[beeme1mr]

Would a list of flag keys be enough sufficient?

[amahiwa-jp]

I believe so. Getting a list of variations would be also nice, but a list of flag keys is a good start.

[thomaspoignant]

having a way to retrieve the list of flag keys can also be used for detecting stale flags in the code.

[beeme1mr]

Hey @maxveldink, could you start a spec proposal? There seems to be sufficient demand for this feature. We should avoid adding a bulk evaluation API but providing a way to access the available flag keys seems valuable.

[maxveldink]

Will do! I'll try to draft it tomorrow or early next week.

[maxveldink]

Started the proposal in this PR.