Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize rack middleware #329

Closed
35 of 36 tasks
tombruijn opened this issue Aug 4, 2017 · 12 comments
Closed
35 of 36 tasks

Standardize rack middleware #329

tombruijn opened this issue Aug 4, 2017 · 12 comments
Assignees

Comments

@tombruijn
Copy link
Member

tombruijn commented Aug 4, 2017

There are problems with mounting grape (and other) apps on Rails apps. This creates two transactions and flip-flops between the two integrations. To make sure only one transaction gets created standardize the rack middleware integration.

TODO

Rack middlewares to refactor

Features

Update docs

Could unblock

@tombruijn tombruijn added this to the Ruby gem 2.4 milestone Aug 4, 2017
@tombruijn
Copy link
Member Author

Waiting for a Grape app where we can reproduce the original problem with

@tombruijn tombruijn changed the title Standardize rack middleware 🐞 Standardize rack middleware Oct 27, 2017
@tombruijn tombruijn changed the title 🐞 Standardize rack middleware Standardize rack middleware May 15, 2018
@tombruijn tombruijn added the Epic label Jun 5, 2018
@matsimitsu
Copy link
Member

We're going to take a day to dive into this issue and create a concrete plan for this, with examples of how the new API is going to change.

@julik
Copy link
Contributor

julik commented Feb 16, 2024

I've given this some thinking: the Rack-derived middleware (integrations) are very similar to one another, so in terms of code reuse they could be brought together or at least inherit from the same superclass (GenericInstrumentation I'd say). In terms of making this more robust, I see a few distinct use cases:

  • Rails app which has a Rack or Sinatra app mounted as an engine in routes.rb - the sidekiqs, mission controls, admins of various kinds etc.
  • Rails app which has a Rack or Sinatra app mounted at the root in config.ru - this does get done sometimes to ensure that there is 0 rails overhead on a particular URL
  • Rack app which uses Rack builder in config.ru and builds a tree of URLs using map and run

In all of those cases it is possible that such a "matroska" (or "umblerella" or whatever) Rack app will actually have multiple instances of the Appsignal middleware active, within the same call to call. For example, a Sinatra app might have Sinatra integration configured, and Rails app might have the Rails integration configured and so on. So whichever nesting is applied, the first order of business would be to ensure that regardless of the number of these instrumentations which get mounted there is only 1 Appsignal transaction used for 1 unit of work (web request), and it gets "centrally" created and centrally completed, once.

This should be very easy to do, provided that 1 decision gets made. If there is nesting between apps, which app wins on setting the action, controller name, path and the params? Do we preserve what gets set in the innermost middleware or we allow the outermost middleware to overwrite instead?

I think all of the above could be achieved in a backwards-compatible way and will also avoid flipflopping. Wdyt @tombruijn ?

@tombruijn
Copy link
Member Author

This should be very easy to do, provided that 1 decision gets made. If there is nesting between apps, which app wins on setting the action, controller name, path and the params? Do we preserve what gets set in the innermost middleware or we allow the outermost middleware to overwrite instead?

I'd say this would be "won" by the most nested app. That is the actual handler of the request. We already use helpers like set_action_if_nil in all our middlewares, that only set the action name if it wasn't set before. I think we can use the same strategy here.

For such nested apps the instrumentation it should not create a new transaction for the nested apps, but use the current transaction, created by the outer most instrumentation. Preferrably, it wouldn't log this warning, so some "check if there is a current AppSignal transaction already" might need to be done.

Some other thoughts I remember we had were:

In the Rails integration, we insert our middleware in a spot that we can rescue and report errors, but ideally we'd be one of the first, if not the first, middleware in the stack. That way we can report the performance of other middlewares too, and the request's status code. We might need to report errors in a different way (maybe the Rails error handler can be used for this?) or the middlware we have now would only report errors with this change.

@julik
Copy link
Contributor

julik commented Feb 19, 2024

So far I was thinking we could then do the following:

  • We set the transaction that got created into env["appsignal.transaction"]. If there are multiple instances of the Rack instrumentation mounted into the tree (be it Sinatra, vanilla Rack, whatever) they would reuse that transaction
  • We do not create a new txn if there already is a env["appsignal.transaction"] and we do not apply the body wrapping. We do, however, set the metadata for the transaction inside ensure of call if the fields are empty
  • Only the outermost middleware would apply a BodyWrapper

I don't quite understand the idea of "instrumenting Rack middleware". If there is one Rack middleware mounted at the outermost possible point, it would already monitor everything happening inside the call of all the things upstream from it, as well as the serving out of the body (via BodyWrapper). Middleware is not straightforward to trace just as a "stack" because every middleware may wrap the response body and return it up the chain, the processing of its body will be deferred. So if instrumenting particular pieces of middleware is desired, I wonder how that would look in, say, the waterfall view 🤔

There is one thing here which is not going to be very straightforward, and that is the handling of filtered params. As it stands now, the "outermost" middleware would produce the request that the txn gets created with. That request would then likely be a Rack request (if the user is not careful enough to mount the Rails integration specifically), and the Rack request does not have filtered_params. That would mean that depending on the order in which things get mounted/added to the stack, filtered params could inadvertently get revealed. I am almost certain a workaround here will be needed because otherwise this creates a security risk for users at the slightest misconfiguration. What we could do is provide some kind of "wrapper" which would only resolve filtered_params at the outermost level of the stack and if (and only if) there is an ActionDispatch request object somewhere (it would be findable in the Rack env).

For the moment what I'm imagining is that we can either

  • Keep the disparate Rack-related middleware classes, just unify them a bit. Then they could have all their special options still, they would just abide by the "reuse the outermost transaction" rule
  • Stuff all the functionality into one and the same Rack middleware, it would just scan the env in a more rigorous way and examine the env and the output headers, and then fill in with the first available option (i.e. scan in order for action name through Appsignal-specific stuff, then Sinatra-specific-stuff, then Rails-specific-stuff etc.)

Either option seems fine tbh, it's just that there will be more deprecations with the latter.

The rest seems trivial.

@julik
Copy link
Contributor

julik commented Feb 20, 2024

Will pick this up once #1037 is in.

@tombruijn
Copy link
Member Author

I don't quite understand the idea of "instrumenting Rack middleware". If there is one Rack middleware mounted at the outermost possible point, it would already monitor everything happening inside the call of all the things upstream from it, as well as the serving out of the body (via BodyWrapper). Middleware is not straightforward to trace just as a "stack" because every middleware may wrap the response body and return it up the chain, the processing of its body will be deferred. So if instrumenting particular pieces of middleware is desired, I wonder how that would look in, say, the waterfall view 🤔

This would be nice for later. We'd like to show each middleware in the timeline, but there's no central place to hook into for that, that I know off. It might require us to make some kind of thin middleware wrapper or some solution like that.

There is one thing here which is not going to be very straightforward, and that is the handling of filtered params. As it stands now, the "outermost" middleware would produce the request that the txn gets created with. That request would then likely be a Rack request (if the user is not careful enough to mount the Rails integration specifically), and the Rack request does not have filtered_params. That would mean that depending on the order in which things get mounted/added to the stack, filtered params could inadvertently get revealed. I am almost certain a workaround here will be needed because otherwise this creates a security risk for users at the slightest misconfiguration. What we could do is provide some kind of "wrapper" which would only resolve filtered_params at the outermost level of the stack and if (and only if) there is an ActionDispatch request object somewhere (it would be findable in the Rack env).

We could have the Rails instrumentation middleware set the params explicitly on the transaction with Appsignal::Transaction#params=. If that's set, it will not check the request object given to the transaction. Would that help?

@backlog-helper

This comment has been minimized.

tombruijn added a commit that referenced this issue Jun 18, 2024
Sinatra apps, mounted in Rails app, would run into the issue that the
Rails middleware has already created a transaction for the request. It
would "force" a new transaction to be made, which loses information from
everything that happened before it.

Previously, before PR #1089, it would also leave a transaction that was
not closed properly. Even with that change, for one request, now two
transactions are created, one for Rails and one for the nested Sinatra
app.

This change reads if there's a current transaction from the request env,
and uses that instead of creating a new one. Some logic in the
Transaction class would read from the request object given to it on
`Transaction.create` to set metadata like parameters, so these need to
be set manually now.

It will also make sure not to close the transaction if one existed
already before this middleware was called.

Part of #329, the Rack middleware refactor.
tombruijn added a commit that referenced this issue Jun 18, 2024
Sinatra apps, mounted in Rails app, would run into the issue that the
Rails middleware has already created a transaction for the request. It
would "force" a new transaction to be made, which loses information from
everything that happened before it.

Previously, before PR #1089, it would also leave a transaction that was
not closed properly. Even with that change, for one request, now two
transactions are created, one for Rails and one for the nested Sinatra
app.

This change reads if there's a current transaction from the request env,
and uses that instead of creating a new one. Some logic in the
Transaction class would read from the request object given to it on
`Transaction.create` to set metadata like parameters, so these need to
be set manually now.

It will also make sure not to close the transaction if one existed
already before this middleware was called.

Part of #329, the Rack middleware refactor.
tombruijn added a commit that referenced this issue Jun 19, 2024
Sinatra apps, mounted in Rails app, would run into the issue that the
Rails middleware has already created a transaction for the request. It
could "force" a new transaction to be made, which loses information from
everything that happened before it.

Previously, before PR #1089, it would also leave a transaction that was
not closed properly. Even with that change, for one request, now two
transactions are created, one for Rails and one for the nested Sinatra
app.

This change reads if there's a current transaction from the request env,
and uses that instead of creating a new one. Some logic in the
Transaction class would read from the request object given to it on
`Transaction.create` to set metadata like parameters, so these need to
be set manually now.

It will also make sure not to close the transaction if one existed
already before this middleware was called.

Part of #329, the Rack middleware refactor.
tombruijn added a commit that referenced this issue Jun 24, 2024
Add the EventHandler to the default Sinatra instrumentation. This will
have it report more of the request runtime.

It will also report the `response_status` tag and metric, as reported by
the EventHandler.

This is compatible with other instrumentations using the EventHandler or
an AbstractMiddleware subclass.

Part of #329
tombruijn added a commit that referenced this issue Jun 25, 2024
Add the EventHandler to the default Sinatra instrumentation. This will
have it report more of the request runtime.

It will also report the `response_status` tag and metric, as reported by
the EventHandler.

This is compatible with other instrumentations using the EventHandler or
an AbstractMiddleware subclass.

Part of #329
tombruijn added a commit that referenced this issue Jun 25, 2024
As part of #329, update the Hanami integration to use Rack middleware
and the EventHandler to instrument requests made to Hanami apps. This
standardizes the instrumentation as much as possible between Rack apps
and minimizes our reliance on monkeypatches.

The only monkeypatch that remains is setting the action name to the
Action class name. I have found no other way yet to fetch this metadata
from the request metadata, environment or the Hanami router.

Part of #329
Mostly solves #911
tombruijn added a commit that referenced this issue Jun 25, 2024
As part of #329, update the Hanami integration to use Rack middleware
and the EventHandler to instrument requests made to Hanami apps. This
standardizes the instrumentation as much as possible between Rack apps
and minimizes our reliance on monkeypatches.

The only monkeypatch that remains is setting the action name to the
Action class name. I have found no other way yet to fetch this metadata
from the request metadata, environment or the Hanami router.

Part of #329
Mostly solves #911
tombruijn added a commit that referenced this issue Jun 25, 2024
As part of #329, update the Hanami integration to use Rack middleware
and the EventHandler to instrument requests made to Hanami apps. This
standardizes the instrumentation as much as possible between Rack apps
and minimizes our reliance on monkeypatches.

The only monkeypatch that remains is setting the action name to the
Action class name. I have found no other way yet to fetch this metadata
from the request metadata, environment or the Hanami router.

Part of #329
Mostly solves #911
tombruijn added a commit that referenced this issue Jun 25, 2024
As part of #329, update the Hanami integration to use Rack middleware
and the EventHandler to instrument requests made to Hanami apps. This
standardizes the instrumentation as much as possible between Rack apps
and minimizes our reliance on monkeypatches.

The only monkeypatch that remains is setting the action name to the
Action class name. I have found no other way yet to fetch this metadata
from the request metadata, environment or the Hanami router.

Part of #329
Mostly solves #911
tombruijn added a commit that referenced this issue Jun 25, 2024
As part of #329, update the Hanami integration to use Rack middleware
and the EventHandler to instrument requests made to Hanami apps. This
standardizes the instrumentation as much as possible between Rack apps
and minimizes our reliance on monkeypatches.

The only monkeypatch that remains is setting the action name to the
Action class name. I have found no other way yet to fetch this metadata
from the request metadata, environment or the Hanami router.

Part of #329
Mostly solves #911
tombruijn added a commit that referenced this issue Jun 25, 2024
As part of #329, update the Hanami integration to use Rack middleware
and the EventHandler to instrument requests made to Hanami apps. This
standardizes the instrumentation as much as possible between Rack apps
and minimizes our reliance on monkeypatches.

The only monkeypatch that remains is setting the action name to the
Action class name. I have found no other way yet to fetch this metadata
from the request metadata, environment or the Hanami router.

Part of #329
Mostly solves #911
tombruijn added a commit that referenced this issue Jun 27, 2024
When AppSignal is already active, do not start AppSignal again.
This is a good precaution as it prevents boot loops, when AppSignal is
started twice with different configurations.

This will improve support for nested Hanami applications, when they're
mounted in another frameworks we support, like Rails or Sinatra.

This is similar to PR #1105

Part of #329
tombruijn added a commit that referenced this issue Jun 27, 2024
When AppSignal is already active, do not start AppSignal again.
This is a good precaution as it prevents boot loops, when AppSignal is
started twice with different configurations.

This will improve support for nested Hanami applications, when they're
mounted in another frameworks we support, like Rails or Sinatra.

This is similar to PR #1105

Part of #329
tombruijn added a commit that referenced this issue Jun 28, 2024
When AppSignal is already active, do not start AppSignal again.
This is a good precaution as it prevents boot loops, when AppSignal is
started twice with different configurations.

This will improve support for nested Hanami applications, when they're
mounted in another frameworks we support, like Rails or Sinatra.

This is similar to PR #1105

Part of #329
tombruijn added a commit that referenced this issue Jul 2, 2024
Apply the same logic as we have in the AbstractMiddleware for webmachine
apps. When a parent transaction is active, use that and don't create a
new one. More importantly, don't close the active transaction.

The webmachine instrumentation can't use the AbstractMiddleware, or any
middleware, as this discouraged by the webmachine project. In practice I
don't see this happening, but if someone happens to add an AppSignal
EventHandler to the middleware stack in `config.ru`, we now support it.

Part of #329
tombruijn added a commit that referenced this issue Jul 2, 2024
Apply the same logic as we have in the AbstractMiddleware for webmachine
apps. When a parent transaction is active, use that and don't create a
new one. More importantly, don't close the active transaction.

The webmachine instrumentation can't use the AbstractMiddleware, or any
middleware, as this discouraged by the webmachine project. In practice I
don't see this happening, but if someone happens to add an AppSignal
EventHandler to the middleware stack in `config.ru`, we now support it.

Part of #329
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants