Skip to content
This repository has been archived by the owner on Feb 28, 2022. It is now read-only.

Commit

Permalink
feat(pipeline): Introduced explicit, named extension points
Browse files Browse the repository at this point in the history
The previous iteration of pipeline extensibility exposed an implementation detail (the function
name) to enable extensibility, implicitly making each step in the pipeline a potential extension
point, and worse, part of the API contract. This change introduces an explicit extension point
marker, called `ext`. It can be used in two ways: (a) by setting the `ext` property of the function
that you want to make an extension point to the name of the extension point (b) by calling the
`ext(name)` chain method on `Pipeline`, which will set the extension point name for the last
function added to the pipeline.
  • Loading branch information
trieloff committed Mar 19, 2019
1 parent 591e5df commit 461d3f6
Show file tree
Hide file tree
Showing 9 changed files with 57 additions and 19 deletions.
14 changes: 8 additions & 6 deletions src/defaults/html.pipe.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,11 @@ const { cache, uncached } = require('../html/shared-cache');
const embeds = require('../html/find-embeds');

/* eslint no-param-reassign: off */
/* eslint newline-per-chained-call: off */

function hascontent({ content }) {
return !(content !== undefined && content.body !== undefined);
}

const htmlpipe = (cont, payload, action) => {
action.logger = action.logger || log;
Expand All @@ -40,8 +45,7 @@ const htmlpipe = (cont, payload, action) => {
pipe
.every(dump).when(() => !production())
.every(validate).when(() => !production())
.before(fetch)
.when(({ content }) => !(content !== undefined && content.body !== undefined))
.before(fetch).when(hascontent)
.before(parse)
.before(embeds)
.before(smartypants)
Expand All @@ -53,12 +57,10 @@ const htmlpipe = (cont, payload, action) => {
.before(responsive)
.once(cont)
.after(type('text/html'))
.after(cache)
.when(uncached)
.after(cache).when(uncached)
.after(key)
.after(debug)
.after(flag)
.when(esi) // flag ESI when there is ESI in the response
.after(flag).when(esi).ext('esi') // flag ESI when there is ESI in the response
.error(selectStatus(production()));

action.logger.log('debug', 'Running HTML pipeline');
Expand Down
2 changes: 2 additions & 0 deletions src/html/fetch-markdown.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,5 +107,7 @@ async function fetch(
}
}

fetch.ext = 'fetch';

This comment has been minimized.

Copy link
@koraa

koraa Mar 19, 2019

Contributor

It seems we now have three ways of defining a function name; alias, ext and name (the default one).
IMO, if we really want to store the name of the pipeline step inside the function, we really could just explicitly overwrite the name property as I have done in sequence.js…

IMO there still is a more fundamental underlying issue:
Specifying that the pipeline steps need to have their name in a property is relatively unusual, which means whether we use ext or .name, this is probably something new helix devs and external devs will run into when they write their first pipeline steps or when they refactor some functions. If we do not check for this it might actually lead to a lot of devs complaining pipeline extensions do not work, unless they have red the fine print in the api documentation specifying that the pipeline step's name is in .ext…

I think we can avoid this issue altogether and create a much smoother development experience by altering .before, .once, .after, … and the extension points to just explicitly take the name of the pipeline step to add…

There are two more reasons why I personally would prefer using an explicit name:

  • It enables the use of anonymous functions (which especially would be useful in short pipelines & pipeline extensions)
  • I think it would be an advantage if the name of a pipeline step could be different for different pipelines; just for the sake of argument, imagine a pipeline with multiple parsing steps; e.g. a pug pipeline; the parse step would parse the pug template and in a further step we would parse inline markdown; this step could be called parse-markdown and might reuse the markdown parser pipeline step…

module.exports = fetch;
module.exports.uri = uri;
3 changes: 3 additions & 0 deletions src/html/flag-esi.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,6 @@ function flag() {
module.exports = {
esi, flag,
};

// make flag esi a pipeline extension point
module.exports.flag.ext = 'flag-esi';
2 changes: 2 additions & 0 deletions src/html/get-metadata.js
Original file line number Diff line number Diff line change
Expand Up @@ -166,4 +166,6 @@ function getmetadata({ content: { sections = [] } }, { logger }) {
return { content: { meta: {} } };
}

getmetadata.ext = 'meta';

module.exports = getmetadata;
2 changes: 2 additions & 0 deletions src/html/make-html.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,6 @@ function html({ content: { mdast }, request }, { logger, secrets }) {
return { content };
}

html.ext = 'html';

module.exports = html;
2 changes: 2 additions & 0 deletions src/html/parse-markdown.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,6 @@ function parse({ content: { body = '' } = {} }, { logger }) {
return { content: { mdast } };
}

parse.ext = 'parse';

module.exports = parse;
28 changes: 17 additions & 11 deletions src/pipeline.js
Original file line number Diff line number Diff line change
Expand Up @@ -115,21 +115,14 @@ class Pipeline {
* @param {integer} before - where to insert the new function (true = before, false = after)
*/
this.attach.generic = (name, f, before = true) => {
const offset = before ? 0 : 2;
// see the describe function below for the generated names
// we parse the extracted name to get the matching extension
// point
const re = new RegExp(`^.*\\:${name} from `);

// find the index of the function where the resolved alias
const offset = before ? 0 : 1;
// find the index of the function where the resolved ext name
// matches the provided name by searching the list of pre and
// post functions
const foundpres = this._pres
.filter(pre => pre.alias)
.findIndex(pre => re.test(pre.alias));
.findIndex(pre => pre && pre.ext && pre.ext === name);
const foundposts = this._posts
.filter(post => post.alias)
.findIndex(post => re.test(post.alias));
.findIndex(post => post && post.ext && post.ext === name);

// if something has been found in either lists, insert the
// new function into the list, with the correct offset
Expand All @@ -139,6 +132,9 @@ class Pipeline {
if (foundposts !== -1) {
this._posts.splice(foundposts + offset, 0, f);
}
if (foundpres === -1 && foundposts === -1) {
this._action.logger.warn(`Unknown extension point ${name}`);
}
};
/**
* Registers an extension to the pipeline. The function `f` will be run in
Expand Down Expand Up @@ -194,6 +190,15 @@ class Pipeline {
return this;
}

/**
* Declares the last function that has been added to be a named extension point
* @param {string} name - name of the new extension point
*/
ext(name) {
this._last.slice(-1).pop().ext = name;
return this;
}

/**
* Adds a condition to the previously defined `pre` or `post` function. The previously defined
* function will only be executed if the predicate evaluates to something truthy or returns a
Expand Down Expand Up @@ -222,6 +227,7 @@ class Pipeline {
return args[0];
};
wrappedfunc.alias = lastfunc.alias;
wrappedfunc.ext = lastfunc.ext;
this._last.push(wrappedfunc);
} else {
throw new Error('when() needs function to operate on.');
Expand Down
11 changes: 9 additions & 2 deletions test/testHTML.js
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ ${content.document.body.innerHTML}`,

let calledfoo = false;
let calledbar = false;
let calledbaz = false;
function foo() {
assert.equal(calledfoo, false, 'foo has not yet been called');
assert.equal(calledbar, false, 'bar has not yet been called');
Expand All @@ -178,6 +179,10 @@ ${content.document.body.innerHTML}`,
calledbar = true;
}

function baz() {
calledbaz = true;
}

function shouttitle(p) {
return {
content: {
Expand All @@ -191,10 +196,11 @@ ${content.document.body.innerHTML}`,
};

myfunc.after = {
flag: bar,
esi: bar,
// after the metadata has been retrived, make sure that
// the title is being shouted
getmetadata: shouttitle,
meta: shouttitle,
never: baz,
};

const res = await pipe(
Expand All @@ -213,6 +219,7 @@ ${content.document.body.innerHTML}`,

assert.equal(calledfoo, true, 'foo has been called');
assert.equal(calledbar, true, 'bar has been called');
assert.equal(calledbaz, false, 'baz has never been called');

assert.ok(res.response.body.match(/HELLO WORLD/));
});
Expand Down
12 changes: 12 additions & 0 deletions test/testPipeline.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ describe('Testing Pipeline', () => {
order.push('four');
};

// inject explicit extension points
[first, second, third, fourth].forEach((f) => {
// eslint-disable-next-line no-param-reassign
f.ext = f.name;
});

const pipe = new Pipeline({ logger })
.before(second)
.once(() => {
Expand Down Expand Up @@ -108,6 +114,12 @@ describe('Testing Pipeline', () => {
order.push('four');
};

// inject explicit extension points
[first, second, third, fourth].forEach((f) => {
// eslint-disable-next-line no-param-reassign
f.ext = f.name;
});

const middle = function middle() {
order.push('middle');
};
Expand Down

0 comments on commit 461d3f6

Please sign in to comment.