Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runTime Discovery Service (TDS) #6708

Closed
htuch opened this issue Apr 25, 2019 · 1 comment · Fixed by #7251
Closed

runTime Discovery Service (TDS) #6708

htuch opened this issue Apr 25, 2019 · 1 comment · Fixed by #7251
Assignees
@htuch
Labels
enhancement Feature requests. Not bugs or questions. no stalebot Disables stalebot from closing an issue
Milestone
1.11.0

Comments

@htuch
Copy link
Member

htuch commented Apr 25, 2019

This issue tracks providing runtime configuration via a pub-sub xDS API called TDS. The idea is that any runtime feature should also be configurable via xDS (and bootstrap statically).

I think a good approach to this might be to define the runtime canonically in protobuf. It would still retain its existing tree hierarchy and types, but the implementation would derive these from the protobuf definition. That way we can easily return a runtime update over TDS or place it in bootstrap.

Another alternative would be to make the TDS/boostrap interface just a key/value map, where runtime settings can be updated via application of the new settings. This would lose some of structural typing advantages of the canonical proto solution, but would be very easy to make work today.
@htuch htuch added the enhancement Feature requests. Not bugs or questions. label Apr 25, 2019
@htuch htuch self-assigned this Apr 25, 2019
@mattklein123 mattklein123 added this to the 1.11.0 milestone Apr 27, 2019
htuch added a commit to htuch/envoy that referenced this issue Apr 30, 2019
@htuch
runtime: support bootstrap runtime configuration.
f892e49 
As a first step towards TDS, add support for proto/JSON representations of the runtime virtual file
system and plumb this into bootstrap.

"numerator" and "denominator" are now reserved keywords and can't be used in runtime keys; this
simplifies parsing significantly. It is not expected that these are widely used, due to the
difficulty of ensuring atomicity if representing fractions, but this is technically a runtime API
breaking change.

Risk level: Medium (due to the reserved word changes).
Testing: Unit and server tests added.

Part of envoyproxy#6708

Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch htuch mentioned this issue Apr 30, 2019
Merged
htuch added a commit that referenced this issue May 6, 2019
@htuch
runtime: support bootstrap runtime configuration. (#6759)
eb699ab 
As a first step towards TDS, add support for proto/JSON representations of the runtime virtual file
system and plumb this into bootstrap.

"numerator" and "denominator" are now reserved keywords and can't be used in runtime keys; this
simplifies parsing significantly. It is not expected that these are widely used, due to the
difficulty of ensuring atomicity if representing fractions, but this is technically a runtime API
breaking change.

Risk level: Medium (due to the reserved word changes).
Testing: Unit and server tests added.

Part of #6708

Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch htuch mentioned this issue May 23, 2019
Merged
@stale
Copy link

stale bot commented May 27, 2019

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

@stale stale bot added the stale stalebot believes this issue/PR has not been touched recently label May 27, 2019
@mattklein123 mattklein123 added the no stalebot Disables stalebot from closing an issue label May 28, 2019
@stale stale bot removed the stale stalebot believes this issue/PR has not been touched recently label May 28, 2019
htuch added a commit that referenced this issue May 30, 2019
@htuch
runtime: flexible layering configuration. (#7032)
4dafba6 
To support TDS and to provide operator flexibility in how it relates to
existing runtime layering, this PR introduces support for explicit
control over layering. This eliminates the distinguished role of
subdirectory and override_subdirectory, for example, in favor of
arbitrary disk overlay layers.

This PR deprecates the existing runtime configuration in favor of the
new layered variant.

As a bonus, the PR fixes some of the runtime stats and related docs.

Risk level: Low
Testing: various unit tests (including server_test) additions.

Part of #6708.

Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch htuch mentioned this issue May 30, 2019
Merged
htuch added a commit that referenced this issue May 30, 2019
@htuch
tds: bootstrap and API configuration protos. (#7105)
ad2cacc 
This PR provides the TDS API config. The implementation and docs will be
added in a followup, the idea is to unblock some projects that depend on
the details here.

Risk level: Low
Testing: @envoy_api build

Part of #6708

Signed-off-by: Harvey Tuch <htuch@google.com>
htuch added a commit to htuch/envoy that referenced this issue Jun 7, 2019
@htuch
config: subscription factory dependency injection.
6b2391b 
While working on TDS, it became apparent that the xDS subscription tests are a bit conflated today,
since they include both testing of the xDS resource specific subscription and also things like the
REST fetcher, etc. This is because the subscription factory is a static, rather than a mockable
dependency of subscriptions. This PR transforms subscription factory to an instance object.

The PR is quite massive, but it has big pay-off in terms of reducing the boiler plate in xDS tests,
separating concerns and making it easier to add new xDS types. Given the recent additions of SRDS,
VHDS, TDS and FCDS, this seems worth it.

A number of xDS tests have been modified to work with the new pattern. Some tests and properties
that are orthogonally tested in subscription_factory_test or the gRPC/REST subscription tests are no
longer tested in the resource specified tests.

Some general absl::string_view goodness plumbing was also needed due to the new string_view-based
interface.

Risk level: Low
Testing: Modified a number of resource specified xDS unit tests.

Relates to envoyproxy#6708.

Signed-off-by: Harvey Tuch <htuch@google.com>
This was referenced Jun 7, 2019
Merged
Closed
htuch added a commit that referenced this issue Jun 11, 2019
@htuch
config: subscription factory dependency injection. (#7200)
b3f848d 
While working on TDS, it became apparent that the xDS subscription tests are a bit conflated today,
since they include both testing of the xDS resource specific subscription and also things like the
REST fetcher, etc. This is because the subscription factory is a static, rather than a mockable
dependency of subscriptions. This PR transforms subscription factory to an instance object.

The PR is quite massive, but it has big pay-off in terms of reducing the boiler plate in xDS tests,
separating concerns and making it easier to add new xDS types. Given the recent additions of SRDS,
VHDS, TDS and FCDS, this seems worth it.

A number of xDS tests have been modified to work with the new pattern. Some tests and properties
that are orthogonally tested in subscription_factory_test or the gRPC/REST subscription tests are no
longer tested in the resource specified tests.

Some general absl::string_view goodness plumbing was also needed due to the new string_view-based
interface.

Risk level: Medium
Testing: Modified a number of resource specified xDS unit tests.

Relates to #6708.

Signed-off-by: Harvey Tuch <htuch@google.com>
htuch added a commit to htuch/envoy that referenced this issue Jun 12, 2019
@htuch
tds: runTime Discovery Service implementation.
ff70cb4 
Fixes envoyproxy#6708.

Risk level: Low
Testing: Unit and integration tests added.

Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch htuch mentioned this issue Jun 12, 2019
Merged
mattklein123 pushed a commit that referenced this issue Jun 18, 2019
@htuch @mattklein123
rtds: Runtime Discovery Service implementation. (#7251)
07da334 
Fixes #6708.

Risk level: Low
Testing: Unit and integration tests added.

Signed-off-by: Harvey Tuch <htuch@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@htuch htuch

Labels
enhancement Feature requests. Not bugs or questions. no stalebot Disables stalebot from closing an issue
Projects
None yet
Milestone
1.11.0
Development

Successfully merging a pull request may close this issue.

rtds: Runtime Discovery Service implementation. htuch/envoy
2 participants
@mattklein123 @htuch