Request/response validators

[leastprivilege]

What should we add or change to make your life better?

YARP has request/response transforms. They are useful for what they are.

They do not have the concept of failing a request or response when certain validation criteria are not satisfied. It would be nice to have a similar way to plugin validators that are able to return error responses to the caller.

Why is this important to you?

Two use cases we have

1. we want to make sure a certain OAuth access token is available to forward to a remote API. If that token is not available, YARP should return a 401 straight away without proxying to the backend
2. require the existence of certain headers (e.g. for antiforgery protection). Again the request should fail immediately.

I know that this could be achieved via middleware - but a mechanism similar to transforms would be less intrusive from configuration point of view.

[davidfowl]

I know that this could be achieved via middleware - but a mechanism similar to transforms would be less intrusive from configuration point of view.

How would it differ from middleware?

[leastprivilege]

It's a different configuration style - instead of adding middleware to the pipeline, you could register the validator with the config system. Just like a transform.

[davidfowl]

Transforms give you more context (the outgoing request). I guess I'm asking for more details here. What would a request validator look like that would do the 2 things you have above? It would be good to throw up a strawman as an example.

[leastprivilege]

So what I would like to do is this:

a) attach some metadata to a route or cluster config
https://github.com/DuendeSoftware/BFF/blob/c8038a2db8205628966152e1bc7f4e0b6f20bb41/samples/JS.Yarp/Startup.cs#L36

b) get invoked at runtime to be able to inspect the incoming request

c) fail the request if necessary

We do this today for access tokens using a transform. But I think the transform is not supposed to fail. Hence the idea of a "validator".

For antiforgery protection we need to fail - this is why we had to use a middleware to implement it.

So a variation of that could be to add the ability for a request transform to return an error to the caller.

[MihaZupan]

Related request: #1670

[samsp-msft]

See #1709 for details of a more extensible config pattern for routes & clusters.

This scenario could be a good example of YARP extension middleware that uses config extensibility to accomplish its goals. The middleware could be supplied as a pre-built library. The consuming YARP app would need to register it and put it in the YARP pipeline.

@Tratcher - should there be a way for extensions to self-register for the YARP pipeline? Today you either go with the default pipeline or define your own. For the default pipeline case, should there be a way for an extension to self-register as running before/after default pipeline stages. Then as a consumer you don't need to figure out the ordering yourself.

[leastprivilege]

I understand that this can be all done with middleware. It is just a different model - and for creating reusable blocks - for transforms we can package it up like this:

services.AddReverseProxy().AddMyExtensions();

which is nicer than teaching the end-user how to wire up middleware the right way.

As I said before - if a transform would have the ability to fault the request and control the response message, this would be very powerful.

[davidfowl]

Transforms have more context and run at a specific time withing the proxy pipeline, I'm wondering why that applies here.

[leastprivilege]

All I am saying is that there are right now 2 fundamentally different approaches. And there are situations where during a transform you realize that the request is invalid - hence the idea of a formal validation step - or the transform being able to fail.

[Tratcher]

It would be possible to add this capability to request transforms. We already have something similar for response transforms:

reverse-proxy/src/ReverseProxy/Transforms/ResponseTransformContext.cs

Lines 31 to 35 in fab6c93

	/// <summary>
	/// Set to true if the proxy should exclude the body and trailing headers when proxying this response.
	/// Defaults to false.
	/// </summary>
	public bool SuppressResponseBody { get; set; }

[karelz]

Triage: We have seen more than 1 ask for it -- e.g. #1724
We should try it in 2.0.

[Tratcher]

Design questions:

• Should this be built into transforms or something separate?
  • Doing something in transforms would be convenient for these cases where the transform discovers the problem.
• Defining validators in config might not have any overlap with existing transforms?
  • What conditions do we need to check?
    • Header values
    • Route values
    • Query values
  • What response fields do we need to set?
    • Status, headers, body
    • I don't think the transforms key-value schema is complex enough to generate arbitrary responses, especially for headers.
• SuppressResponseBody relies on ValueTask<bool> HttpTransformer.TransformResponseAsync.
  • Doing the same thing with ValueTask HttpTransformer.TransformRequestAsync would be a breaking change to the return value. However, the bool would mean something different.
  • We've gotten some feedback that the bool return value is confusing.
  • Adding bool RequestTransformContext.RespondWithoutProxying is easy/non-breaking.
  • Changing ValueTask RequestTransform.ApplyAsync to return a bool to allow short circuiting
  • What if we just check for HttpResponse.HasStarted? It's not intuitive because just setting a status code won't trigger the response, but it doesn't require any API changes.
  • Or if the status code has changed?
  • Should we run response transforms when short circuiting? We can't really if the response has already started.

[Tratcher]

Similar request: #1893, how to define routes in config that don't proxy.