feat(*): Adds initial config interface implementation #4

thomastaylor312 · 2023-11-22T20:46:38Z

This is the most basic functionality needed to have a shared config interface that all components could use. It's goal is simplicity. There are many things we could add/polish, and all of those are called out in the documentation. For reference, this is what we are using in wasmCloud, but we aren't married to the interface as is. What we want is the ability for any runtime to be able to provide configuration to the components it is running in a standardized way. That will make it possible to actually have the cross runtime/platform experience that components promise

thomastaylor312 · 2023-11-22T20:47:18Z

Please note that I'll be intermittently available the rest of the week and completely unavailable next week, so if I don't respond, that is why!

lukewagner

From a first glance, this looks reasonable to me; nice! I included some nits, but I don't feel super-strongly about them, so happy to discuss.

wit/runtime_config.wit

wit/world.wit

Mossaka

Thanks for contributing. Mostly LGTM just added one comment on the README.

README.md

lann

I briefly looked at this months ago (#2) and came to the conclusion that - for the use cases I was coming up with - there wasn't much benefit to a separate runtime config interface over just implementing as a KV store.

As @Mossaka asked inline I think it would be really helpful to understand concrete use cases here and in particular where those use cases wouldn't be well served by wasi-keyvalue.

lann · 2023-11-27T21:13:18Z

wit/runtime_config.wit

+    /// Gets a single opaque config value set at the given key if it exists
+    get: func(key: string) -> result<option<list<u8>>, config-error>;
+    /// Gets a list of all set config data
+    get-all: func() -> result<list<tuple<string, list<u8>>>, config-error>;


What are some use cases for get-all?

Some apps and libraries iterate over environment variables to look for prefix matches, e.g. "find all variables starting with FOO_BAR_" as a way of encoding a variable-length map or list. So perhaps this would make it easier to port such software.

Yeah @dicej had it right. There is also a use case where you want to fetch all the config once and then iterate over it in memory rather than doing individual fetches

thomastaylor312 · 2023-12-05T22:32:59Z

Great question @lann! So there are definitely some distinct differences between KV and config. But more importantly, the ultimate goal here are interfaces that increase interoperability. This is the absolute bare minimum interface that heads towards that goal and allows a component to indicate very clearly that "I am looking for config" rather than a datastore. Some additional points:

Having to do config via a KV means you have to have a KV store available. Some platforms and use cases might not have that immediately available. It also leads to an awkward experience: "To configure your component, write the following keys to Redis before running it."
Related to the last point, but there is a difference between a data store and configuration. Yes, config may possibly be derived from a remote database or tool like vault, but more often than not it is also combined with host specific things, for example what k8s passes to pods by default. There is also a reason why things like ConfigMaps and external config exists for K8s and Nomad. You could theoretically have people store stuff directly in etcd/consul and access that, but config is something that most applications need.
With those last points in mind, it is also important to note that this is meant to be the least common denominator for interoperability. Not everything is going to use wasi:cli/environment with environment variables (but it could be easily virtualized out with a component that converts from env vars to config values). wasmCloud is obviously my easiest example here, but any system that is running untrusted code or not as a FaaS type system could either not expose or have other security reasons for not wanting to expose environment variables to the system, instead delegating that layer of control to the host.

This is the most basic functionality needed to have a shared config interface that all components could use. It's goal is simplicity. There are many things we could add/polish, and all of those are called out in the documentation. For reference, this is what we are using in wasmCloud, but we aren't married to the interface as is. What we want is the ability for any runtime to be able to provide configuration to the components it is running in a standardized way. That will make it possible to actually have the cross runtime/platform experience that components promise Signed-off-by: Taylor Thomas <taylor@cosmonic.com>

lhmzhou

LGTM

thomastaylor312 · 2024-01-05T19:27:45Z

Hey all (particularly @Mossaka and @devigned)! Now that we're in the new year, I'd like to circle back to this and see if we can get it approved or have a concrete list of things to change. Please let me know your thoughts!

Mossaka

Asked some questions, otherwise it LGTM

Mossaka · 2024-01-05T19:45:25Z

README.md

+   all sorts of crazy things with configuration values! Calling entirely different branches of code,
+   setting upstream URLs or services, configuring the number of threads to use, etc.
+2. toggling on/off feature flags
+3. A/B testing

 and many more use cases.


Is secrets management in-scope?

This is an interesting discussion. For now, I'm going to say no. But I think that access to secret values could be through this interface. Implementors (generally hosts) would need to make sure that secret values are injected into the available config

Mossaka · 2024-01-05T19:49:21Z

README.md

+   all sorts of crazy things with configuration values! Calling entirely different branches of code,
+   setting upstream URLs or services, configuring the number of threads to use, etc.
+2. toggling on/off feature flags
+3. A/B testing

 and many more use cases.


Some advanced features I've seen from runtime config providers like Google Cloud's Runtime Configurator are:

versioning,

hierarchy

watching

Although this spec does not have to include any of above features but I would like to see they are mentioned somewhere in the spec.

Do you want those called out as "not addressed right now, but maybe in the future" or do you want it mentioned in some other way?

Calling out as not addressed right now is good.

I am going to merge this PR. This can be added later.

lukewagner reviewed Nov 27, 2023

View reviewed changes

wit/runtime_config.wit Outdated Show resolved Hide resolved

wit/runtime_config.wit Outdated Show resolved Hide resolved

wit/world.wit Outdated Show resolved Hide resolved

Mossaka reviewed Nov 27, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

lann reviewed Nov 27, 2023

View reviewed changes

thomastaylor312 force-pushed the feat/pull-interface branch from 09092fa to c768db1 Compare December 6, 2023 17:11

lhmzhou approved these changes Dec 12, 2023

View reviewed changes

Mossaka approved these changes Jan 5, 2024

View reviewed changes

Mossaka merged commit 7207661 into WebAssembly:main Jan 12, 2024

thomastaylor312 deleted the feat/pull-interface branch January 17, 2024 17:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(*): Adds initial config interface implementation #4

feat(*): Adds initial config interface implementation #4

thomastaylor312 commented Nov 22, 2023

thomastaylor312 commented Nov 22, 2023

lukewagner left a comment

Mossaka left a comment

lann left a comment •

edited

Loading

lann Nov 27, 2023

dicej Nov 28, 2023

thomastaylor312 Dec 5, 2023

thomastaylor312 commented Dec 5, 2023

lhmzhou left a comment

thomastaylor312 commented Jan 5, 2024

Mossaka left a comment

Mossaka Jan 5, 2024

thomastaylor312 Jan 9, 2024

Mossaka Jan 5, 2024

thomastaylor312 Jan 9, 2024

Mossaka Jan 12, 2024

feat(*): Adds initial config interface implementation #4

feat(*): Adds initial config interface implementation #4

Conversation

thomastaylor312 commented Nov 22, 2023

thomastaylor312 commented Nov 22, 2023

lukewagner left a comment

Choose a reason for hiding this comment

Mossaka left a comment

Choose a reason for hiding this comment

lann left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomastaylor312 commented Dec 5, 2023

lhmzhou left a comment

Choose a reason for hiding this comment

thomastaylor312 commented Jan 5, 2024

Mossaka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lann left a comment •

edited

Loading