Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gatsby-plugin-functions): Add the ability to run functions locally and on Gatsby Cloud #30192

Merged
merged 76 commits into from
Apr 16, 2021
Merged
Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
098a209
Add new plugin
sidharthachatterjee Nov 16, 2020
7576e2a
Add functions plugin by default
sidharthachatterjee Nov 16, 2020
df0496f
Add stub
sidharthachatterjee Nov 16, 2020
5d0c3e8
Bump up package.json
sidharthachatterjee Nov 16, 2020
9febad5
wip
sidharthachatterjee Nov 24, 2020
0597a5f
Compile all functions found in build
sidharthachatterjee Nov 24, 2020
2536188
Add plugin by default only if GATSBY_EXPERIMENT_FUNCTIONS is set
sidharthachatterjee Nov 24, 2020
29159df
Move to public for now
sidharthachatterjee Nov 25, 2020
eec2a09
Move to functions dir
sidharthachatterjee Nov 25, 2020
6470ee4
Run a dev server for functions
sidharthachatterjee Nov 30, 2020
9635aa1
wip
sidharthachatterjee Nov 30, 2020
4a59a47
Move to onPreBootstrap
sidharthachatterjee Jan 5, 2021
85d7cfa
Add target node
abhiaiyer91 Jan 11, 2021
ef07abd
Bump up package.json
sidharthachatterjee Jan 7, 2021
72877bc
Bump up package.json
sidharthachatterjee Jan 12, 2021
daf3b04
Add support for env vars
sidharthachatterjee Feb 19, 2021
32bd3d6
Publish 0.1.0-4
sidharthachatterjee Feb 19, 2021
a2ae24e
Bump up peerDependencies
sidharthachatterjee Mar 2, 2021
c1a2a05
Update yarn lock
sidharthachatterjee Mar 11, 2021
3d9ecbc
Use a parameter in express middleware and ignore extension (#30222)
sidharthachatterjee Mar 12, 2021
6067919
Merge remote-tracking branch 'upstream/master' into feat/functions
Mar 16, 2021
2aacbcc
Publish 0.1.0-6
sidharthachatterjee Mar 16, 2021
1587dc2
Set a default functions directory path (#30277)
sidharthachatterjee Mar 17, 2021
f0ab110
feat(functions): Add body parser
julienp Mar 21, 2021
b5324f9
Require files from the .cache directory so user can run es/commonjs (…
abhiaiyer91 Mar 25, 2021
21e1841
Publish 0.1.0-7
sidharthachatterjee Mar 25, 2021
6ce15fa
Prefer directory from program in state
sidharthachatterjee Apr 2, 2021
b2c4aa8
Log invocations as verbose
sidharthachatterjee Apr 2, 2021
bd0acc5
Add timeout constant
sidharthachatterjee Apr 2, 2021
4cfde03
Add internal plugin
sidharthachatterjee Apr 2, 2021
cc98e76
Load functions internal plugin
sidharthachatterjee Apr 2, 2021
17730bd
Load functions during initialize and stick em in redux
sidharthachatterjee Apr 2, 2021
80a9882
Merge branch 'master' into feat/functions
sidharthachatterjee Apr 7, 2021
56176d3
Clean up comments
sidharthachatterjee Apr 7, 2021
eace7e8
Move initialization code back to plugin
sidharthachatterjee Apr 7, 2021
5c1f96e
Add experiment for Gatsby Functions
sidharthachatterjee Apr 7, 2021
6b8d414
Revert yarn changes
sidharthachatterjee Apr 7, 2021
b325d12
Remove plugin package
sidharthachatterjee Apr 7, 2021
88d76a1
Do not extract comments
sidharthachatterjee Apr 7, 2021
311c9ec
Update yarn.lock
sidharthachatterjee Apr 7, 2021
d554093
Update packages/gatsby/src/internal-plugins/functions/gatsby-node.ts
julienp Apr 13, 2021
63bc54c
chore(functions): Remove console.logs
julienp Apr 13, 2021
f08d0ee
chore(functions): Remove unused urlResolve import
julienp Apr 13, 2021
5d6d072
Merge remote-tracking branch 'upstream/master' into feat/functions
Apr 13, 2021
7fbac4a
get internal plugin functions working + some cleanups
KyleAMathews Apr 13, 2021
4af22f9
Fix regex per @jamo's suggestion + update yarn.lock
KyleAMathews Apr 13, 2021
324bd99
remove accidentally committed file
KyleAMathews Apr 13, 2021
92e05c9
globby isn't a dependency of gatsby
KyleAMathews Apr 13, 2021
2163f08
fix(functions): End request with status 500 on function error
julienp Apr 13, 2021
aad744d
Hot reload functions
KyleAMathews Apr 13, 2021
c5c61ec
Load env variables from .env.* files
KyleAMathews Apr 13, 2021
aaf53f9
Resolve result of function so we can catch errors from async functions
KyleAMathews Apr 13, 2021
d2d7af0
Log webpack warnings/errors when compiling to the terminal
KyleAMathews Apr 13, 2021
02fcfbe
Log how long function took to execute
KyleAMathews Apr 13, 2021
4bec919
reload watch when .env files change & use all process.env vars
KyleAMathews Apr 13, 2021
ac9e55d
Allow for arbitrarily deep functions
KyleAMathews Apr 13, 2021
a26ac14
Detect new functions and incorporate
KyleAMathews Apr 13, 2021
5cc3661
Log when functions rebuild or we restart the watcher
KyleAMathews Apr 13, 2021
ae8f46f
List functions on dev-404-page
KyleAMathews Apr 13, 2021
0336739
Only show new API instructions when trying to reach an API route + ad…
KyleAMathews Apr 13, 2021
7282d1f
Fix linting
KyleAMathews Apr 14, 2021
6caff93
put new code behind conditional
KyleAMathews Apr 14, 2021
4b5aa4e
Properly close webpack watcher + handle deleting functions
KyleAMathews Apr 14, 2021
10226e4
Ignore non-js extensions (so make typescript work)
KyleAMathews Apr 14, 2021
1ee2d24
Write out manifest file so can serve functions from 'gatsby serve'
KyleAMathews Apr 14, 2021
4a9fffd
Address @lekoart's comments
KyleAMathews Apr 15, 2021
3cba097
Add default type for SiteFunction & fix small bugs when site doesn't …
KyleAMathews Apr 15, 2021
9c164f7
Add initial suite of tests
KyleAMathews Apr 15, 2021
771fb62
Update snapshots
KyleAMathews Apr 15, 2021
b2f2af2
Fix more tests
KyleAMathews Apr 15, 2021
73dceb4
Add functions tests to circleci setup
KyleAMathews Apr 15, 2021
10cd117
Only parse once
KyleAMathews Apr 16, 2021
42de7d5
Small tweaks
KyleAMathews Apr 16, 2021
8405b31
add comment
KyleAMathews Apr 16, 2021
af36e09
Fix typescript support
KyleAMathews Apr 16, 2021
2b7c7cf
Add tests for apis with special characters h/t @lekoarts
KyleAMathews Apr 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,13 @@ jobs:
- store_artifacts:
path: integration-tests/images/__diff_output__

integration_tests_functions:
executor: node
steps:
- e2e-test:
test_path: integration-tests/functions
test_command: yarn test

e2e_tests_path-prefix:
<<: *e2e-executor
environment:
Expand Down Expand Up @@ -610,6 +617,8 @@ workflows:
<<: *e2e-test-workflow
- integration_tests_images:
<<: *e2e-test-workflow
- integration_tests_functions:
<<: *e2e-test-workflow
- integration_tests_gatsby_cli:
requires:
- bootstrap
Expand Down
1 change: 1 addition & 0 deletions integration-tests/functions/.env.development
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pickle=word
3 changes: 3 additions & 0 deletions integration-tests/functions/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
node_modules/
.cache/
public
54 changes: 54 additions & 0 deletions integration-tests/functions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
<p align="center">
<a href="https://www.gatsbyjs.com/?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter">
<img alt="Gatsby" src="https://www.gatsbyjs.com/Gatsby-Monogram.svg" width="60" />
</a>
</p>
<h1 align="center">
Gatsby minimal starter
</h1>

## 🚀 Quick start

1. **Create a Gatsby site.**

Use the Gatsby CLI to create a new site, specifying the minimal starter.

```shell
# create a new Gatsby site using the minimal starter
npm init gatsby
```

2. **Start developing.**

Navigate into your new site’s directory and start it up.

```shell
cd my-gatsby-site/
npm run develop
```

3. **Open the code and start customizing!**

Your site is now running at http://localhost:8000!

Edit `src/pages/index.js` to see your site update in real-time!

4. **Learn more**

- [Documentation](https://www.gatsbyjs.com/docs/?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter)

- [Tutorials](https://www.gatsbyjs.com/tutorial/?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter)

- [Guides](https://www.gatsbyjs.com/tutorial/?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter)

- [API Reference](https://www.gatsbyjs.com/docs/api-reference/?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter)

- [Plugin Library](https://www.gatsbyjs.com/plugins?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter)

- [Cheat Sheet](https://www.gatsbyjs.com/docs/cheat-sheet/?utm_source=starter&utm_medium=readme&utm_campaign=minimal-starter)

## 🚀 Quick start (Gatsby Cloud)

Deploy this starter with one click on [Gatsby Cloud](https://www.gatsbyjs.com/cloud/):

[<img src="https://www.gatsbyjs.com/deploynow.svg" alt="Deploy to Gatsby Cloud">](https://www.gatsbyjs.com/dashboard/deploynow?url=https://github.com/gatsbyjs/my-gatsby-site)
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`functions can parse different ways of sending data form data 1`] = `
Object {
"a": "form-data",
}
`;

exports[`functions can parse different ways of sending data form parameters 1`] = `
Object {
"a": "form parameters",
}
`;

exports[`functions can parse different ways of sending data json body 1`] = `
Object {
"a": "json",
}
`;

exports[`functions can parse different ways of sending data query string 1`] = `
Object {
"amIReal": "true",
}
`;

exports[`response formats returns json correctly 1`] = `
Object {
"amIJSON": true,
}
`;

exports[`response formats returns json correctly 2`] = `
Object {
"access-control-allow-origin": "*",
"connection": "close",
"content-length": "16",
"content-type": "application/json; charset=utf-8",
"etag": "W/\\"10-R6td1pV+B+Xz9CJkNeaEI2kP+QY\\"",
"vary": "Accept-Encoding",
"x-powered-by": "Express",
}
`;

exports[`response formats returns text correctly 1`] = `"I am typescript"`;

exports[`response formats returns text correctly 2`] = `
Object {
"access-control-allow-origin": "*",
"connection": "close",
"content-length": "15",
"content-type": "text/html; charset=utf-8",
"etag": "W/\\"f-zwggT56l/fnWBm4dI0PkA4E3i4E\\"",
"vary": "Accept-Encoding",
"x-powered-by": "Express",
}
`;

exports[`routing secondary-level API 1`] = `"I am at a secondary-level"`;

exports[`routing secondary-level API 2`] = `"I am another sub-directory function"`;

exports[`routing secondary-level API with index.js 1`] = `"I am an index.js in a sub-directory!"`;

exports[`routing top-level API 1`] = `"I am at the top-level"`;

exports[`typescript typescript functions work 1`] = `"I am typescript"`;
3 changes: 3 additions & 0 deletions integration-tests/functions/__tests__/fixtures/function-a.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export default function Function(req, res) {
res.send(`hi`)
}
1 change: 1 addition & 0 deletions integration-tests/functions/__tests__/fixtures/test.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
hi friends
205 changes: 205 additions & 0 deletions integration-tests/functions/__tests__/functions.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
const fetch = require(`node-fetch`)
const execa = require(`execa`)
const fs = require(`fs-extra`)
const path = require(`path`)
const FormData = require("form-data")

describe(`routing`, () => {
test(`top-level API`, async () => {
const result = await fetch(
`http://localhost:8000/api/top-level`
).then(res => res.text())

expect(result).toMatchSnapshot()
})
test(`secondary-level API`, async () => {
const result = await fetch(
`http://localhost:8000/api/a-directory/function`
).then(res => res.text())

expect(result).toMatchSnapshot()
})
test(`secondary-level API with index.js`, async () => {
const result = await fetch(
`http://localhost:8000/api/a-directory`
).then(res => res.text())

expect(result).toMatchSnapshot()
})
test(`secondary-level API`, async () => {
const result = await fetch(
`http://localhost:8000/api/dir/function`
).then(res => res.text())

expect(result).toMatchSnapshot()
})
})

describe(`environment variables`, () => {
test(`can use inside functions`, async () => {
const result = await fetch(
`http://localhost:8000/api/env-variables`
).then(res => res.text())

expect(result).toEqual(`word`)
})
})

describe(`typescript`, () => {
test(`typescript functions work`, async () => {
const result = await fetch(
`http://localhost:8000/api/i-am-typescript`
).then(res => res.text())

expect(result).toMatchSnapshot()
})
})

describe(`response formats`, () => {
test(`returns json correctly`, async () => {
const res = await fetch(`http://localhost:8000/api/i-am-json`)
const result = await res.json()

const { date, ...headers } = Object.fromEntries(res.headers)
expect(result).toMatchSnapshot()
expect(headers).toMatchSnapshot()
})
test(`returns text correctly`, async () => {
const res = await fetch(`http://localhost:8000/api/i-am-typescript`)
const result = await res.text()

const { date, ...headers } = Object.fromEntries(res.headers)
expect(result).toMatchSnapshot()
expect(headers).toMatchSnapshot()
})
})

describe(`functions can send custom statuses`, () => {
test(`can return 200 status`, async () => {
const res = await fetch(`http://localhost:8000/api/status`)

expect(res.status).toEqual(200)
})

test(`can return 404 status`, async () => {
const res = await fetch(`http://localhost:8000/api/status?code=404`)

expect(res.status).toEqual(404)
})

test(`can return 500 status`, async () => {
const res = await fetch(`http://localhost:8000/api/status?code=500`)

expect(res.status).toEqual(500)
})
})

describe(`functions can parse different ways of sending data`, () => {
test(`query string`, async () => {
const result = await fetch(
`http://localhost:8000/api/parser?amIReal=true`
).then(res => res.json())

expect(result).toMatchSnapshot()
})

test(`form parameters`, async () => {
const { URLSearchParams } = require("url")
const params = new URLSearchParams()
params.append("a", `form parameters`)
const result = await fetch(`http://localhost:8000/api/parser`, {
method: `POST`,
body: params,
}).then(res => res.json())

expect(result).toMatchSnapshot()
})

test(`form data`, async () => {
const FormData = require("form-data")

const form = new FormData()
form.append("a", `form-data`)
const result = await fetch(`http://localhost:8000/api/parser`, {
method: `POST`,
body: form,
}).then(res => res.json())

expect(result).toMatchSnapshot()
})

test(`json body`, async () => {
const body = { a: `json` }
const result = await fetch(`http://localhost:8000/api/parser`, {
method: `POST`,
body: JSON.stringify(body),
headers: { "Content-Type": "application/json" },
}).then(res => res.json())

expect(result).toMatchSnapshot()
})

// test(`file in multipart/form`, async () => {
LekoArts marked this conversation as resolved.
Show resolved Hide resolved
// const { readFileSync } = require("fs")

// const file = readFileSync(path.join(__dirname, "./fixtures/test.txt"))

// const form = new FormData()
// form.append("file", file)
// const result = await fetch(`http://localhost:8000/api/parser`, {
// method: `POST`,
// body: form,
// }).then(res => res.json())

// console.log({ result })

// expect(result).toMatchSnapshot()
// })

// test(`stream a file`, async () => {
// const { createReadStream } = require("fs")

// const stream = createReadStream(path.join(__dirname, "./fixtures/test.txt"))
// const res = await fetch(`http://localhost:8000/api/parser`, {
// method: `POST`,
// body: stream,
// })

// console.log(res)

// expect(result).toMatchSnapshot()
// })
})

// TODO figure out why this gets into endless loops
// describe.only(`hot reloading`, () => {
// const fixturesDir = path.join(__dirname, `fixtures`)
// const apiDir = path.join(__dirname, `../src/api`)
// beforeAll(() => {
// try {
// fs.unlinkSync(path.join(apiDir, `function-a.js`))
// } catch (e) {
// // Ignore as this should mostly error with file not found.
// // We delete to be sure it's not there.
// }
// })
// afterAll(() => {
// fs.unlinkSync(path.join(apiDir, `function-a.js`))
// })

// test(`new function`, cb => {
// fs.copySync(
// path.join(fixturesDir, `function-a.js`),
// path.join(apiDir, `function-a.js`)
// )
// setTimeout(async () => {
// const result = await fetch(
// `http://localhost:8000/api/function-a`
// ).then(res => res.text())

// console.log(result)
// expect(result).toMatchSnapshot()
// cb()
// }, 400)
// })
// })
9 changes: 9 additions & 0 deletions integration-tests/functions/gatsby-config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
module.exports = {
flags: {
FUNCTIONS: true,
},
siteMetadata: {
title: "functions",
},
plugins: [],
}
13 changes: 13 additions & 0 deletions integration-tests/functions/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
module.exports = {
testPathIgnorePatterns: [
`/node_modules/`,
`__tests__/fixtures`,
`.cache`,
`src/test`,
`src/api`,
],
watchPathIgnorePatterns: ["src/api", ".cache"],
transform: {
"^.+\\.[jt]sx?$": `<rootDir>../../jest-transformer.js`,
},
}
Loading