ref(nextjs): Inject init code in _app and API routes

[lobsterkatie]

TL;DR:

This PR changes how we include the user's sentry.server.config.js and sentry.client.config.js code (which is what calls Sentry.init()) in server and browser bundles, in order both to fix apps deployed to Vercel and to eliminate some semi-gross hacks included in the current implementation.

Background

Because of the way nextjs apps behave when deployed to Vercel, our current system of initializing the SDK during server start up doesn't work for API routes. (Vercel splits such routes into AWS lambdas, and in order to keep the size of each lambda small, it uses a stripped-down version of the server when it does so. As a result, half of our current monkeypatching doesn't even run, including the part which starts the SDK.)

Further, the current system is ugly in two ways. Neither one of them is a blocker, but they're both less than ideal.

• On the server side, we currently create a separate bundle out of the user's sentry.server.config.js file. In order to be sure we can find it later, at build time we log this bundle's location to .env.local (which contains environment variables that nextjs automatically makes available at runtime). This isn't great, because it forces us to worry about whether they already have such a file (so we can merge rather than overwrite it), and if they don't, creates a surprise file in the user's project directory with no explanation.

• On the client side, we currently inject the user's sentry.client.config.js file into the main bundle. This is fine, except in cases where nextjs's backwards-compatibility-preserving code ends up overwriting the change. (See here for more details and the hack we use to work around the problem.) The hack we have works, but isn't pretty.

This PR fixes all three of those problems.

Changes

It turns out the easiest way to fix things is to avoid the server question (stripped down or not) altogether. This PR therefore:

• Stops creating a separate server-init bundle, writing its location to disk, and monkeypatching the nextjs server to dynamically require it when it starts up. Instead, it injects the initializing code directly into the API route files. By making sure to add it specifically to the beginning of each API entry point, we can ensure that the SDK will be up and running by the time any code in the route file runs.

• Stops injecting the client-init code into main and instead injects it into pages/_app, which is the container wrapping all user-created pages, and whose code therefore runs before theirs does, similarly guaranteeing the SDK will have started up by the time theirs runs.

• Does the same thing (injecting init code into pages/app) on the server side. Though the SDK isn't yet totally functional in non-API routes, this at least means that unhandled errors and promise rejections can be captured.

Notes

• Because we're no longer injecting into nextjs framework code (the server itself and, on the client side, the main bundle), we no longer need to protect against publicRuntimeConfig being undefined, since it's loaded when the framework starts up. By the time _app is rendered, or API handlers are run, that start-up process has long since completed.

• In my testing on Vercel, I sometimes ran into the situation where transactions for API routes appeared not to be getting successfully sent to Sentry. After a great deal of experimentation, I finally discovered that in these cases, the sending wasn't blocked but merely delayed - the transaction would in fact send itself, but not until the next time the same route was hit (provided it was soon enough that the route's lambda function hadn't been garbage collected for being idle too long; in those cases, the transaction was simply lost). Thanks to @kamilogorek, I tracked it down to the AWS callbackWaitsForEmptyEventLoop option, which nextjs sets to false by default, thereby causing the lambda to shut down as soon as the API response is finished. Though it's possible to override that setting, it will require reformatting withSentry to itself be a lambda, and in order to not complicate things further, that work will happen in a future PR. [UPDATE: Updating withSentry to return a lambda directly turned out to be quite complicated, so in the end we're instead monkeypatching res.end() to wait to fire until all events have been sent to Sentry.)

• This PR also doesn't fix the problem of subsequent requests to other API routes somehow thinking they're a continuation of the same trace. That problem remains a mystery (though I suspect it has something to do with scope bleed), and will also need to be solved in a future PR. [UPDATE: Though it doesn't solve all of our scope problems, this PR fixes this false positive continuation issue.)

Fixes #3723
Fixes #3683
Fixes #3777
Fixes #3648
Fixes #3688
Fixes #3691
Fixes #3748

[github-actions]

size-limit report

Path	Size
@sentry/browser - CDN Bundle (gzipped)	21.43 KB (+0.01% 🔺)
@sentry/browser - Webpack	22.45 KB (0%)
@sentry/react - Webpack	22.49 KB (0%)
@sentry/browser + @sentry/tracing - CDN Bundle (gzipped)	28.93 KB (+0.01% 🔺)

[iker-barriocanal]

Everything else LGTM, and note that the missing PR description may answer my early questions.

[Vadorequest]

High five tracking down the issues with AWS Lambda, they're not easy eggs to catch. 👍