Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Selective parsers with child logger #1741

Merged
merged 32 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
585c333
Breaking: changing beforeUpload to a middleware.
RobinTail May 7, 2024
ddc7efc
The concept draft (not complete).
RobinTail May 7, 2024
0eb73dd
Ref: using app.use(path) for parsers required for the certain endpoint.
RobinTail May 7, 2024
bf4b737
Adjusting the tests.
RobinTail May 7, 2024
81ec8a6
Ref: extracting rawMover, with tests, moving test.
RobinTail May 7, 2024
9a3b21c
Avoid using parsers for OPTIONS method.
RobinTail May 7, 2024
43d8e17
Revert "Avoid using parsers for OPTIONS method."
RobinTail May 7, 2024
d1cd670
Remove todo.
RobinTail May 7, 2024
4a6f4e9
FEAT: using the request level logger on parsers and middlewares from …
RobinTail May 7, 2024
44fc73e
Reverting #1733 and cc3fff8 (partially).
RobinTail May 7, 2024
3686e3b
Ref: makeCommonEntities() returns handlers equipped with request leve…
RobinTail May 7, 2024
400f2b8
Ref: extracting createUploadMiddleware().
RobinTail May 7, 2024
186c614
Test for the new helper.
RobinTail May 7, 2024
b43484e
Moving the logger middleware to the top, combining last two .use().
RobinTail May 7, 2024
111903a
Adjusting ServerConfig: fix defaults, inline type for beforeRouting.
RobinTail May 7, 2024
2f7fb7e
REF: transforming beforeUpload into a similar type as it was before, …
RobinTail May 7, 2024
a61f511
Moving logger into the metaSymbol prop of locals.
RobinTail May 7, 2024
2da2c9f
Update src/config-type.ts
RobinTail May 7, 2024
facb055
Reducing diff.
RobinTail May 7, 2024
e7fa19a
Ref: extracting type Parsers.
RobinTail May 7, 2024
82988c7
Changelog: update for 19.0.0.
RobinTail May 8, 2024
e2b3ad8
REF: placing parsers into the app.METHOD call because app.use treats …
RobinTail May 8, 2024
0575f52
Todo for a future.
RobinTail May 8, 2024
5f3ecb3
Revert exporting ChildLoggerProvider.
RobinTail May 8, 2024
86e9fe0
REF: createUploadParsers() to take all jobs related to establish uplo…
RobinTail May 8, 2024
f5073bc
Ref: shortening constructing parsers dict.
RobinTail May 8, 2024
2125400
Changelog: notice on revert.
RobinTail May 8, 2024
8296f66
Changelog: migration advice on beforeUpload.
RobinTail May 8, 2024
5974700
Ref: reducing imports in server helpers.
RobinTail May 8, 2024
8034fbf
Ref: minor arrangement to commons made by makeCommonEntities().
RobinTail May 8, 2024
e7a7d7c
Minor: moving mocks import higher in test.
RobinTail May 8, 2024
f5ef3de
Minor: naming.
RobinTail May 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions src/config-type.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export type TagsConfig<TAG extends string> = Record<
string | { description: string; url?: string }
>;

type ChildLoggerProvider = (params: {
export type ChildLoggerProvider = (params: {
request: Request;
parent: AbstractLogger;
}) => AbstractLogger | Promise<AbstractLogger>;
Expand Down Expand Up @@ -75,6 +75,16 @@ export interface CommonConfig<TAG extends string = string> {
tags?: TagsConfig<TAG>;
}

export type BeforeUpload = (params: {
request: Request;
logger: AbstractLogger;
}) => void | Promise<void>;

type BeforeRouting = (params: {
app: IRouter;
logger: AbstractLogger;
}) => void | Promise<void>;

type UploadOptions = Pick<
fileUpload.Options,
| "createParentPath"
Expand All @@ -95,24 +105,18 @@ type UploadOptions = Pick<
* */
limitError?: Error;
/**
* @desc A code to execute before connecting the upload middleware.
* @desc It can be used to connect a middleware that restricts the ability to upload.
* @desc A handler to execute before uploading — it can be used for restrictions by throwing an error.
* @default undefined
* @example ({ app }) => { app.use( ... ); }
* @example ({request}) => { throw createHttpError(403, "Not authorized"); }
RobinTail marked this conversation as resolved.
Show resolved Hide resolved
* */
beforeUpload?: AppExtension;
beforeUpload?: BeforeUpload;
RobinTail marked this conversation as resolved.
Show resolved Hide resolved
};

type CompressionOptions = Pick<
compression.CompressionOptions,
"threshold" | "level" | "strategy" | "chunkSize" | "memLevel"
>;

type AppExtension = (params: {
app: IRouter;
logger: AbstractLogger;
}) => void | Promise<void>;

export interface ServerConfig<TAG extends string = string>
extends CommonConfig<TAG> {
/** @desc Server configuration. */
Expand All @@ -127,13 +131,13 @@ export interface ServerConfig<TAG extends string = string>
jsonParser?: RequestHandler;
/**
* @desc Enable or configure uploads handling.
* @default false
* @default undefined
* @requires express-fileupload
* */
upload?: boolean | UploadOptions;
/**
* @desc Enable or configure response compression.
* @default false
* @default undefined
* @requires compression
*/
compression?: boolean | CompressionOptions;
Expand All @@ -152,7 +156,7 @@ export interface ServerConfig<TAG extends string = string>
* @default undefined
* @example ({ app }) => { app.use('/docs', swaggerUi.serve, swaggerUi.setup(swaggerDocument)); }
* */
beforeRouting?: AppExtension;
beforeRouting?: BeforeRouting;
};
/** @desc Enables HTTPS server as well. */
https?: {
Expand Down
19 changes: 12 additions & 7 deletions src/routing.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
import { IRouter } from "express";
import { IRouter, RequestHandler } from "express";
import { CommonConfig } from "./config-type";
import { ContentType } from "./content-type";
import { DependsOnMethod } from "./depends-on-method";
import { AbstractEndpoint } from "./endpoint";
import { AbstractLogger } from "./logger";
import { metaSymbol } from "./metadata";
import { walkRouting } from "./routing-walker";
import { ServeStatic } from "./serve-static";
import { LocalResponse } from "./server-helpers";

export interface Routing {
[SEGMENT: string]: Routing | DependsOnMethod | AbstractEndpoint | ServeStatic;
Expand All @@ -15,25 +18,27 @@ export const initRouting = ({
rootLogger,
config,
routing,
parsers,
}: {
app: IRouter;
rootLogger: AbstractLogger;
config: CommonConfig;
routing: Routing;
parsers?: Record<ContentType, RequestHandler[]>;
}) =>
walkRouting({
routing,
hasCors: !!config.cors,
onEndpoint: (endpoint, path, method, siblingMethods) => {
app[method](path, async (request, response) => {
const logger = config.childLoggerProvider
? await config.childLoggerProvider({ request, parent: rootLogger })
: rootLogger;
logger.info(`${request.method}: ${path}`);
const middlewares = parsers?.[endpoint.getRequestType()] || [];
if (middlewares.length) {
app.use(path, middlewares);
RobinTail marked this conversation as resolved.
Show resolved Hide resolved
}
app[method](path, async (request, response: LocalResponse) => {
await endpoint.execute({
request,
response,
logger,
logger: response.locals[metaSymbol]?.logger || rootLogger,
config,
siblingMethods,
});
Expand Down
94 changes: 68 additions & 26 deletions src/server-helpers.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import type fileUpload from "express-fileupload";
import { metaSymbol } from "./metadata";
import { AnyResultHandlerDefinition } from "./result-handler";
import { AbstractLogger } from "./logger";
import { CommonConfig } from "./config-type";
import { ErrorRequestHandler, RequestHandler } from "express";
import { BeforeUpload, CommonConfig } from "./config-type";
import { ErrorRequestHandler, RequestHandler, Response } from "express";
import createHttpError, { isHttpError } from "http-errors";
import { lastResortHandler } from "./last-resort";
import { ResultHandlerError } from "./errors";
Expand All @@ -10,16 +12,16 @@ import { makeErrorFromAnything } from "./common-helpers";
interface HandlerCreatorParams {
errorHandler: AnyResultHandlerDefinition;
rootLogger: AbstractLogger;
getChildLogger: CommonConfig["childLoggerProvider"];
}

export type LocalResponse = Response<
unknown,
{ [metaSymbol]?: { logger: AbstractLogger } }
>;

export const createParserFailureHandler =
({
errorHandler,
rootLogger,
getChildLogger,
}: HandlerCreatorParams): ErrorRequestHandler =>
async (error, request, response, next) => {
({ errorHandler, rootLogger }: HandlerCreatorParams): ErrorRequestHandler =>
async (error, request, response: LocalResponse, next) => {
if (!error) {
return next();
}
Expand All @@ -32,26 +34,18 @@ export const createParserFailureHandler =
input: null,
output: null,
options: {},
logger: getChildLogger
? await getChildLogger({ request, parent: rootLogger })
: rootLogger,
logger: response.locals[metaSymbol]?.logger || rootLogger,
});
};

export const createNotFoundHandler =
({
errorHandler,
getChildLogger,
rootLogger,
}: HandlerCreatorParams): RequestHandler =>
async (request, response) => {
({ errorHandler, rootLogger }: HandlerCreatorParams): RequestHandler =>
async (request, response: LocalResponse) => {
const error = createHttpError(
404,
`Can not ${request.method} ${request.path}`,
);
const logger = getChildLogger
? await getChildLogger({ request, parent: rootLogger })
: rootLogger;
const logger = response.locals[metaSymbol]?.logger || rootLogger;
try {
errorHandler.handler({
request,
Expand Down Expand Up @@ -86,9 +80,57 @@ export const createUploadFailueHandler =
export const createUploadLogger = (
logger: AbstractLogger,
): Pick<Console, "log"> => ({
log: (message, ...rest) => {
if (!/not eligible/.test(message)) {
logger.debug(message, ...rest);
}
},
log: logger.debug.bind(logger),
});

export const createUploadMiddleware =
({
options,
uploader,
rootLogger,
beforeUpload,
}: {
options: fileUpload.Options;
uploader: typeof fileUpload;
rootLogger: AbstractLogger;
beforeUpload?: BeforeUpload;
RobinTail marked this conversation as resolved.
Show resolved Hide resolved
}): RequestHandler =>
async (request, response: LocalResponse, next) => {
const logger = response.locals[metaSymbol]?.logger || rootLogger;
try {
await beforeUpload?.({ request, logger });
} catch (error) {
return next(error);
}
uploader({
...options,
abortOnLimit: false,
parseNested: true,
logger: createUploadLogger(logger),
})(request, response, next);
};

export const rawMover: RequestHandler = (req, {}, next) => {
if (Buffer.isBuffer(req.body)) {
req.body = { raw: req.body };
}
next();
};

/** @since v19 prints the actual path of the request, not a configured route */
export const createLoggingMiddleware =
({
rootLogger,
config,
}: {
rootLogger: AbstractLogger;
config: CommonConfig;
}): RequestHandler =>
async (request, response: LocalResponse, next) => {
const logger = config.childLoggerProvider
? await config.childLoggerProvider({ request, parent: rootLogger })
: rootLogger;
logger.info(`${request.method}: ${request.path}`);
response.locals[metaSymbol] = { logger };
next();
};
58 changes: 29 additions & 29 deletions src/server.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
import express from "express";
import express, { RequestHandler } from "express";
import type compression from "compression";
import type fileUpload from "express-fileupload";
import http from "node:http";
import https from "node:https";
import { AppConfig, CommonConfig, ServerConfig } from "./config-type";
import { ContentType } from "./content-type";
import { AbstractLogger, createLogger, isBuiltinLoggerConfig } from "./logger";
import { loadPeer } from "./peer-helpers";
import { defaultResultHandler } from "./result-handler";
import { Routing, initRouting } from "./routing";
import {
createLoggingMiddleware,
createNotFoundHandler,
createParserFailureHandler,
createUploadFailueHandler,
createUploadLogger,
createUploadMiddleware,
rawMover,
} from "./server-helpers";
import { getStartupLogo } from "./startup-logo";

Expand All @@ -25,8 +28,7 @@ const makeCommonEntities = (config: CommonConfig) => {
: config.logger;
rootLogger.debug("Running", process.env.TSUP_BUILD || "from sources");
const errorHandler = config.errorHandler || defaultResultHandler;
const { childLoggerProvider: getChildLogger } = config;
const creatorParams = { errorHandler, rootLogger, getChildLogger };
const creatorParams = { errorHandler, rootLogger };
const notFoundHandler = createNotFoundHandler(creatorParams);
const parserFailureHandler = createParserFailureHandler(creatorParams);
return { rootLogger, errorHandler, notFoundHandler, parserFailureHandler };
Expand All @@ -40,6 +42,10 @@ export const attachRouting = (config: AppConfig, routing: Routing) => {

export const createServer = async (config: ServerConfig, routing: Routing) => {
const app = express().disable("x-powered-by");
const { rootLogger, notFoundHandler, parserFailureHandler } =
makeCommonEntities(config);
app.use(createLoggingMiddleware({ rootLogger, config }));

if (config.server.compression) {
const compressor = await loadPeer<typeof compression>("compression");
app.use(
Expand All @@ -50,46 +56,40 @@ export const createServer = async (config: ServerConfig, routing: Routing) => {
),
);
}
app.use(config.server.jsonParser || express.json());

const { rootLogger, notFoundHandler, parserFailureHandler } =
makeCommonEntities(config);
const parsers: Record<ContentType, RequestHandler[]> = {
json: [config.server.jsonParser || express.json()],
upload: [],
raw: [],
};

if (config.server.upload) {
const uploader = await loadPeer<typeof fileUpload>("express-fileupload");
const { limitError, beforeUpload, ...derivedConfig } = {
const { limitError, beforeUpload, ...options } = {
...(typeof config.server.upload === "object" && config.server.upload),
};
if (beforeUpload) {
beforeUpload({ app, logger: rootLogger });
}
app.use(
uploader({
...derivedConfig,
abortOnLimit: false,
parseNested: true,
logger: createUploadLogger(rootLogger),
}),
parsers.upload.push(
createUploadMiddleware({ uploader, options, rootLogger, beforeUpload }),
);
if (limitError) {
app.use(createUploadFailueHandler(limitError));
parsers.upload.push(createUploadFailueHandler(limitError));
}
}
if (config.server.rawParser) {
app.use(config.server.rawParser);
app.use((req, {}, next) => {
if (Buffer.isBuffer(req.body)) {
req.body = { raw: req.body };
}
next();
});
parsers.raw.push(config.server.rawParser, rawMover);
}
app.use(parserFailureHandler);
if (config.server.beforeRouting) {
await config.server.beforeRouting({ app, logger: rootLogger });
}
initRouting({ app, routing, rootLogger, config });
app.use(notFoundHandler);

initRouting({
app,
routing,
rootLogger,
config,
parsers,
});
app.use(parserFailureHandler, notFoundHandler);

const starter = <T extends http.Server | https.Server>(
server: T,
Expand Down
3 changes: 3 additions & 0 deletions src/testing.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { AbstractEndpoint } from "./endpoint";
import { AbstractLogger } from "./logger";
import { contentTypes } from "./content-type";
import { loadAlternativePeer } from "./peer-helpers";
import { LocalResponse } from "./server-helpers";

/**
* @desc Using module augmentation approach you can set the Mock type of your actual testing framework.
Expand Down Expand Up @@ -54,11 +55,13 @@ export const makeResponseMock = <RES extends Record<string, any>>({
responseMock.writableEnded = true;
return responseMock;
}),
locals: {},
...responseProps,
} as {
writableEnded: boolean;
statusCode: number;
statusMessage: string;
locals: LocalResponse["locals"];
} & Record<
"set" | "setHeader" | "header" | "status" | "json" | "send" | "end",
MockOverrides
Expand Down
2 changes: 1 addition & 1 deletion tests/system/example.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ describe("Example", async () => {
createdAt: "2022-01-22T00:00:00.000Z",
},
});
await waitFor(() => /v1\/user\/:id/.test(out));
await waitFor(() => /v1\/user\/50/.test(out));
RobinTail marked this conversation as resolved.
Show resolved Hide resolved
await waitFor(() => /50, 123, 456/.test(out));
expect(true).toBeTruthy();
});
Expand Down
Loading