Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

draft: Cloudflare works ! 🎉 #618

Merged
merged 2 commits into from
Jul 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 86 additions & 49 deletions cf/polyfills.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,61 +13,69 @@ const IPv4Reg = new RegExp(`^${v4Str}$`)
const v6Seg = '(?:[0-9a-fA-F]{1,4})'
const IPv6Reg = new RegExp(
'^(' +
`(?:${v6Seg}:){7}(?:${v6Seg}|:)|` +
`(?:${v6Seg}:){6}(?:${v4Str}|:${v6Seg}|:)|` +
`(?:${v6Seg}:){5}(?::${v4Str}|(:${v6Seg}){1,2}|:)|` +
`(?:${v6Seg}:){4}(?:(:${v6Seg}){0,1}:${v4Str}|(:${v6Seg}){1,3}|:)|` +
`(?:${v6Seg}:){3}(?:(:${v6Seg}){0,2}:${v4Str}|(:${v6Seg}){1,4}|:)|` +
`(?:${v6Seg}:){2}(?:(:${v6Seg}){0,3}:${v4Str}|(:${v6Seg}){1,5}|:)|` +
`(?:${v6Seg}:){1}(?:(:${v6Seg}){0,4}:${v4Str}|(:${v6Seg}){1,6}|:)|` +
`(?::((?::${v6Seg}){0,5}:${v4Str}|(?::${v6Seg}){1,7}|:))` +
')(%[0-9a-zA-Z-.:]{1,})?$'
`(?:${v6Seg}:){7}(?:${v6Seg}|:)|` +
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some linting issues - I've went ahead and used eslint on this file but no kudos :(

`(?:${v6Seg}:){6}(?:${v4Str}|:${v6Seg}|:)|` +
`(?:${v6Seg}:){5}(?::${v4Str}|(:${v6Seg}){1,2}|:)|` +
`(?:${v6Seg}:){4}(?:(:${v6Seg}){0,1}:${v4Str}|(:${v6Seg}){1,3}|:)|` +
`(?:${v6Seg}:){3}(?:(:${v6Seg}){0,2}:${v4Str}|(:${v6Seg}){1,4}|:)|` +
`(?:${v6Seg}:){2}(?:(:${v6Seg}){0,3}:${v4Str}|(:${v6Seg}){1,5}|:)|` +
`(?:${v6Seg}:){1}(?:(:${v6Seg}){0,4}:${v4Str}|(:${v6Seg}){1,6}|:)|` +
`(?::((?::${v6Seg}){0,5}:${v4Str}|(?::${v6Seg}){1,7}|:))` +
')(%[0-9a-zA-Z-.:]{1,})?$'
)

const textEncoder = new TextEncoder()
export const crypto = {
randomBytes: l => Crypto.getRandomValues(Buffer.alloc(l)),
pbkdf2Sync: async(password, salt, iterations, keylen) => Crypto.subtle.deriveBits(
{
name: 'PBKDF2',
hash: 'SHA-256',
salt,
iterations
},
await Crypto.subtle.importKey(
'raw',
textEncoder.encode(password),
'PBKDF2',
false,
randomBytes: (l) => Crypto.getRandomValues(Buffer.alloc(l)),
pbkdf2Sync: async (password, salt, iterations, keylen) =>
Crypto.subtle.deriveBits(
{
name: 'PBKDF2',
hash: 'SHA-256',
salt,
iterations
},
await Crypto.subtle.importKey(
'raw',
textEncoder.encode(password),
'PBKDF2',
false,
['deriveBits']
),
keylen * 8,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing changed here

['deriveBits']
),
keylen * 8,
['deriveBits']
),
createHash: (type) => ({
update: (x) => ({
digest: () => {
return type === 'sha256'
? Crypto.subtle.digest('SHA-256', x)
: Crypto.subtle.digest('MD5', x)
if (type !== 'sha256')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed MD5 as you asked

throw Error('createHash only supports sha256 on cloudflare.')
porsager marked this conversation as resolved.
Show resolved Hide resolved
if (!(x instanceof Uint8Array))
x = textEncoder.encode(x)
return Crypto.subtle.digest('SHA-256', x)
}
})
}),
createHmac: (type, key) => ({
update: x => ({
digest: async() => Buffer.from(await Crypto.subtle.sign(
'HMAC',
await Crypto.subtle.importKey('raw', key, { name: 'HMAC', hash: 'SHA-256' }, false, ['sign']),
textEncoder.encode(x)
))
update: (x) => ({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linting only

digest: async () =>
Buffer.from(
await Crypto.subtle.sign(
'HMAC',
await Crypto.subtle.importKey(
'raw',
key,
{ name: 'HMAC', hash: 'SHA-256' },
false,
['sign']
),
textEncoder.encode(x)
)
)
})
})
}

export const process = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added at the end of the file because of hrtime polyfill

env: {}
}

export const os = {
userInfo() {
return { username: 'postgres' }
Expand All @@ -81,19 +89,23 @@ export const fs = {
}

export const net = {
isIP: x => RegExp.prototype.test.call(IPv4Reg, x) ? 4 : RegExp.prototype.test.call(IPv6Reg, x) ? 6 : 0,
isIP: (x) =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting only

RegExp.prototype.test.call(IPv4Reg, x)
porsager marked this conversation as resolved.
Show resolved Hide resolved
? 4
: RegExp.prototype.test.call(IPv6Reg, x)
? 6
: 0,
Socket
}

export { setImmediate, clearImmediate }

export const tls = {
connect(x) {
const tcp = x.socket
connect({ socket: tcp, servername }) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Destructuring here 🤔
Not sure if you want to keep this, maybe a merge side-effect. Feel free to keep it or not - this code works (I've added some tests files down this PR)

tcp.writer.releaseLock()
tcp.reader.releaseLock()
tcp.readyState = 'upgrading'
tcp.raw = tcp.raw.startTls({ servername: x.servername })
tcp.raw = tcp.raw.startTls({ servername })
tcp.raw.closed.then(
() => tcp.emit('close'),
(e) => tcp.emit('error', e)
Expand Down Expand Up @@ -133,7 +145,7 @@ function Socket() {
() => {
tcp.readyState !== 'upgrade'
? close()
: (tcp.readyState = 'open', tcp.emit('secureConnect'))
: ((tcp.readyState = 'open'), tcp.emit('secureConnect'))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting

},
(e) => tcp.emit('error', e)
)
Expand All @@ -151,8 +163,7 @@ function Socket() {
}

function close() {
if (tcp.readyState === 'closed')
return
if (tcp.readyState === 'closed') return
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting


tcp.readyState = 'closed'
tcp.emit('close')
Expand All @@ -164,9 +175,7 @@ function Socket() {
}

function end(data) {
return data
? tcp.write(data, () => tcp.raw.close())
: tcp.raw.close()
return data ? tcp.write(data, () => tcp.raw.close()) : tcp.raw.close()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting

}

function destroy() {
Expand All @@ -178,7 +187,7 @@ function Socket() {
try {
let done
, value
while (({ done, value } = await tcp.reader.read(), !done))
while ((({ done, value } = await tcp.reader.read()), !done))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting

tcp.emit('data', Buffer.from(value))
} catch (err) {
error(err)
Expand Down Expand Up @@ -211,3 +220,31 @@ function setImmediate(fn) {
function clearImmediate(id) {
tasks.delete(id)
}

const nowOffset = Date.now()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added hrtime polyfill because it was used in the code (as process.hrtime)

const now = () => Date.now() - nowOffset
const hrtime = (previousTimestamp) => {
const baseNow = Math.floor((Date.now() - now()) * 1e-3)
const clocktime = now() * 1e-3
let seconds = Math.floor(clocktime) + baseNow
let nanoseconds = Math.floor((clocktime % 1) * 1e9)

if (previousTimestamp) {
seconds = seconds - previousTimestamp[0]
nanoseconds = nanoseconds - previousTimestamp[1]
if (nanoseconds < 0) {
seconds--
nanoseconds += 1e9
}
}
return [seconds, nanoseconds]
}
hrtime.bigint = () => {
const time = hrtime()
return BigInt(`${time[0]}${time[1]}`)
}

export const process = {
env: {},
hrtime
}
6 changes: 1 addition & 5 deletions cf/src/connection.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
import { process } from '../polyfills.js'
import { Buffer } from 'node:buffer'
import { setImmediate, clearImmediate } from '../polyfills.js'
import { net } from '../polyfills.js'
import { tls } from '../polyfills.js'
import { crypto } from '../polyfills.js'
import { setImmediate, clearImmediate, process, net, tls, crypto } from '../polyfills.js'
Copy link
Contributor Author

@wackfx wackfx Jul 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworked transpile.cf.js to allow for a single ../polyfills.js import (eslint was not happy about the multiple imports)

import Stream from 'node:stream'

import { stringify, handleValue, arrayParser, arraySerializer } from './types.js'
Expand Down
4 changes: 1 addition & 3 deletions cf/src/index.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
import { process } from '../polyfills.js'
import { os } from '../polyfills.js'
import { fs } from '../polyfills.js'
import { process, os, fs } from '../polyfills.js'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here


import {
mergeUserTypes,
Expand Down
22 changes: 22 additions & 0 deletions cf/test-pages.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
// Add your database url and run this file with
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file gives some instructions to test the code on pages wrangler (notice the pages in the command)

// npx wrangler pages dev ./cf --script-path test-pages.js --compatibility-date=2023-06-20 --log-level=debug --compatibility-flag=nodejs_compat
import postgres from './src/index'
const DATABASE_URL = ''

export default {
async fetch(request, env) {
const url = new URL(request.url);
if (url.pathname.includes('/favicon.ico')) {
return new Response('')
}
if (url.pathname.startsWith('/')) {
const sql = postgres(DATABASE_URL)
const rows = await sql`SELECT table_name FROM information_schema.columns`
return new Response(rows.map((e) => e.table_name).join('\n'))
}

// Otherwise, serve the static assets.
// Without this, the Worker will error and no assets will be served.
return env.ASSETS.fetch(request)
},
}
15 changes: 15 additions & 0 deletions cf/test-worker.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Add your database url and run this file with
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regular wrangler test

// npx wrangler dev ./cf/test-worker.js --compatibility-date=2023-06-20 --log-level=debug --compatibility-flag=nodejs_compat
import postgres from './src/index'
const DATABASE_URL = ''

export default {
async fetch(request, env, ctx) {
if (request.url.includes('/favicon.ico'))
return new Response()

const sql = postgres(DATABASE_URL)
const rows = await sql`SELECT table_name FROM information_schema.columns`
return new Response(rows.map((e) => e.table_name).join('\n'))
},
}
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,4 @@
"pg",
"database"
]
}
}
43 changes: 26 additions & 17 deletions transpile.cf.js
Original file line number Diff line number Diff line change
@@ -1,38 +1,47 @@
import fs from 'fs'
import path from 'path'

const empty = x => fs.readdirSync(x).forEach(f => fs.unlinkSync(path.join(x, f)))
, ensureEmpty = x => !fs.existsSync(x) ? fs.mkdirSync(x) : empty(x)
const empty = (x) =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linter

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you're using the .eslint from this repo?

fs.readdirSync(x).forEach((f) => fs.unlinkSync(path.join(x, f)))
, ensureEmpty = (x) => (!fs.existsSync(x) ? fs.mkdirSync(x) : empty(x))
, root = 'cf'
, src = path.join(root, 'src')

ensureEmpty(src)

fs.readdirSync('src').forEach(name =>
fs.readdirSync('src').forEach((name) =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linter

fs.writeFileSync(
path.join(src, name),
transpile(fs.readFileSync(path.join('src', name), 'utf8'), name, 'src')
)
)

function transpile(x) {
const timers = x.includes('setImmediate')
? 'import { setImmediate, clearImmediate } from \'../polyfills.js\'\n'
: ''

const process = x.includes('process.')
? 'import { process } from \'../polyfills.js\'\n'
: ''
const polyfills = [
Copy link
Contributor Author

@wackfx wackfx Jul 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworked transpile.cf.js to allow for a single ../polyfills.js import (eslint was not happy about the multiple imports)

x.includes('setImmediate') ? ['setImmediate', 'clearImmediate'] : undefined,
x.includes('process') ? ['process'] : undefined,
x.includes('import net from \'net\'') ? ['net'] : undefined,
x.includes('import tls from \'tls\'') ? ['tls'] : undefined,
x.includes('import crypto from \'crypto\'') ? ['crypto'] : undefined,
x.includes('import os from \'os\'') ? ['os'] : undefined,
x.includes('import fs from \'fs\'') ? ['fs'] : undefined
].filter(Boolean).flat()

const buffer = x.includes('Buffer')
? 'import { Buffer } from \'node:buffer\'\n'
: ''

return process + buffer + timers + x
.replace('import net from \'net\'', 'import { net } from \'../polyfills.js\'')
.replace('import tls from \'tls\'', 'import { tls } from \'../polyfills.js\'')
.replace('import crypto from \'crypto\'', 'import { crypto } from \'../polyfills.js\'')
.replace('import os from \'os\'', 'import { os } from \'../polyfills.js\'')
.replace('import fs from \'fs\'', 'import { fs } from \'../polyfills.js\'')
.replace(/ from '([a-z_]+)'/g, ' from \'node:$1\'')
return (
buffer +
// bulk add polyfills
(polyfills.length ? `import { ${polyfills.join(', ')} } from '../polyfills.js'\n` : '') +
x
// cleanup polyfills
.replace('import crypto from \'crypto\'\n', '')
.replace('import net from \'net\'\n', '')
.replace('import tls from \'tls\'\n', '')
.replace('import os from \'os\'\n', '')
.replace('import fs from \'fs\'\n', '')
.replace(/ from '([a-z_]+)'/g, ' from \'node:$1\'')
)
}