Add ability to provide certs for https local dev server

[sdarnell]

What version of Wrangler are you using?

2.1.14

What operating system are you using?

Mac

Describe the Bug

When using wrangler to test workers, I need to use https, and use --local-protocol https, well actually I specify it in the toml with:

[dev]
local_protocol = "https"

The problem is that when running in local mode, miniflare is used and it generates a self signed cert every single time I launch. This message is output each time:

[mf:inf] Generating new self-signed certificate...

A freshly generated self-signed cert every time is awkward because most modern browsers will reject them.

When not in local mode, the wrangler code creates/reads the cert, and will default to ~/.wrangler/local-cert
For my use case I replace the cert in this folder, but I can't do this miniflare when it's running in a temp dir.

Also, I would argue that the two modes should use the same cert, and from a look at the code it looks like miniflare can be passed the path to the cert and key (options https-key and https-cert).

My personal preference would be to be able to optionally be able to pass the cert & key files through to wrangler, and it also forward those on to miniflare in local mode.

[sdarnell]

The flags in miniflare are here: https://github.com/cloudflare/miniflare/blob/master/packages/http-server/src/plugin.ts#L92-L95

[caass]

Thanks for bringing this up! I've added it to our backlog. @mrbbot thoughts?

[sdarnell]

I just saw another potential solution on discord: https://discord.com/channels/595317990191398933/1052656806058528849/threads/1058205277079490620
That is to use Node's NODE_EXTRA_CA_CERTS=file
See https://nodejs.org/api/cli.html#node_extra_ca_certsfile

It may also be possible to use NODE_OPTIONS=--use-openssl-ca
as suggested by https://stackoverflow.com/a/68135600/1129543

However, with the latter I get:

  cause: Error: unable to get local issuer certificate
      at TLSSocket.onConnectSecure (node:_tls_wrap:1530:34)
      at TLSSocket.emit (node:events:390:28)
      at TLSSocket._finishInit (node:_tls_wrap:944:8)
      at TLSWrap.ssl.onhandshakedone (node:_tls_wrap:725:12) {
    code: 'UNABLE_TO_GET_ISSUER_CERT_LOCALLY'

[amery]

it's also important to be able to specify cert/key per instance. overwriting cert.pem and key.pem in ~/.config/.wrangler/local-cert/ doesn't allow using more than one worker simultaneously.
and ideally from the environment like many other CLOUDFLARE_ options

[jabranr]

I faced a similar issue while setting up HTTPS for localhost on the Remix app targeted for CF Pages. I tried setting up the mkcert cert and key files in ~/.config/.wrangler/local-cert/ but it did not make any difference. In fact, the directory did not exist. Ideally, there should be a way to point those cert and key files per project. I also tried keeping the cert.pem and key.pem in the project root directory and then project_root/local-cert/ but neither worked. I even tried logging output from different places in node_modules/wrangler/wrangler-dist/cli.js but in vain. Unable to find the value for CERT_ROOT as mentioned in comments here: https://github.com/cloudflare/workers-sdk/blob/main/packages/wrangler/src/https-options.ts#L18

it's also important to be able to specify cert/key per instance. overwriting cert.pem and key.pem in ~/.config/.wrangler/local-cert/ doesn't allow using more than one worker simultaneously. and ideally from the environment like many other CLOUDFLARE_ options

[jabranr]

it's also important to be able to specify cert/key per instance. overwriting cert.pem and key.pem in ~/.config/.wrangler/local-cert/ doesn't allow using more than one worker simultaneously. and ideally from the environment like many other CLOUDFLARE_ options

I think it is at ~/.wrangler/local-cert/ instead of ~/.config/.wrangler/local-cert/ for MacOS.

[amery]

that's what find ~ -name "*wrangler*" found for me (Ubuntu)

[AlexErrant]

I'm on Ubuntu 20.04.4 and it used to be at ~/.config/.wrangler/local-cert/... but I recently upgraded to Wrangler 2.13.0 and it changed to ~/.wrangler/local-cert/. I'm not sure if 2.13.0 was the cause - I just know that the upgrade and my scripts breaking occurred relatively close in time. There doesn't seem to be any mention of this in the release logs, so I might be crazy.

Edit: On a fresh install of Ubuntu 22.04.2, I installed wrangler 2.15.1 and the config is at ~/.config/.wrangler/local-cert/. So I'm pretty sure it's not the version update causing the path change (unless Cloudflare is playing a really weird game...) I've no idea what determines the directory 😐

Edit2: Okay, as per source code, it depends on your XDG_CONFIG_HOME... unless you already have a dir at the legacy location, ~/.wrangler. I guess something really just borked up in my computer to cause it to resort to the legacy location, and once it's there it's stuck. Some info for noobs like me:

The XDG Base Directory Specification is based on the following concepts... There is a single base directory relative to which user-specific configuration files should be written. This directory is defined by the environment variable $XDG_CONFIG_HOME.

Source.

[jabranr]

FYI I have started looking into this to create a PR. I am thinking that if SSL certs exist at the project root then it should pick those up. I am considering local-cert/ directory at the project root for this. It would fall back to the default behaviour otherwise.

[AlexErrant]

Another issue in the archived repo considered specifying the cert/key paths in the wrangler.toml. Might be worth opening an issue in this repo with the specs of your PR, and maybe asking why the "PR Welcome" tag was removed from the aforementioned issue.

[amery]

Another issue in the archived repo considered specifying the cert/key paths in the wrangler.toml. Might be worth opening an issue in this repo with the specs of your PR, and maybe asking why the "PR Welcome" tag was removed from the aforementioned issue.

I believe priorities should be ENV, wrangler.toml, magic name in the same directory of wrangler.toml, and then magic location in $HOME

[jabranr]

I am struggling a bit with running the code in debug mode. There is no documentation around it either so I have to debug it as part of a Remix CF Pages app. It seems that there was an attempt to bring Miniflare into Wrangler but then Wrangler is still using the Miniflare core package as its dependency for various reasons.

Can someone advise on it?

[jschlesser]

Hi @jabranr ,
I am running into the inconvenience with the new ssl certificates on a reload as well. I think you are on the right track with the miniflare options. I am working with wrangler pages dev (no wrangler.toml support) so I would prefer the ENV route as well. I think you are on the right track with the miniflare options. When this line in the dev/local package evaluates to true, puts miniflare in a mode where it generates self signed certificates. I think that there are quite a few options that will need to be added to cover all the bases. I think that they are here in the miniflare repo.

There seem to be so much surface area where all these options are passed through, I think they all converge in that local package though. However, I think they will have to be enabled everywhere command line options or possibly just in one spot for ENV parameters and or a magic location in the project or $HOME. ENV seems the most flexible with the least effort.

I'm open to helping if you want. I'm working on a sveltekit project on pages dev using oauth (requires ssl for local testing) so I have to re-accept the self signed cert every single code change. I can't use a proxy for a different reason, so I'm really stuck.

[penalosa]

@jspspike does your work capture the issues here?

[jspspike]

My changes shouldn't resolve this entirely. If you change the key.pem and cert.pem files located in the wrangler config directory, those certs should not be regenerated until the cert expires. CERT_EXPIRY_DAYS is currently set to 30 days.

We can leave this issue open as a request to add a dedicated way to pass in a key and cert for the local https server if that is something that is in demand

[AlexErrant]

add a dedicated way to pass in a key and cert for the local https server

Yes, please.

(I don't really care if this issue is used to track this feature request or not - it may necessitate updating the issue's title.)

[jschlesser]

@jspspike Thanks for the changes you have already made.

For whomever decides to pick this up at any time in the future. I'd like to add details for some use cases that I have.

I work on multiple workers based projects and they work with various versions of wrangler. Because of this I tend to not use the global wrangler and have wrangler installed as a dev dependency in my projects. Per project vs global is important to me.

I work on projects that have multiple services running so I have to have multiple wranglers running simultaneously. Having multiple configs even within a single project could be beneficial to me.

I work on projects that have outside dependencies like social logins (GitHub, Google etc...) and many of those services require a full domain to be set up like local.myproject.com for local testing (no localhost or 127.0.0.1) or ci.myproject.com for ci automated testing and local / ci browsers need to be configured to trust those certificates. I really need to bring my own PKI to the table in these scenarios.

There is probably an argument here to just push ssl termination to a reverse proxy but I have run into at least one instance in the past where that was an issue. I think it was a pages dev project with svelte-kit (ultimately vite behind svelte-kit) that was having issues.

If it had to be a single solution within workers-sdk, I would choose command line options for user generated certificates. I feel like most scenarios could be covered by those two options combined.

[paulrostorp]

I've got a PR for this opened, it seems to have gone unnoticed....
#4475

@caass @jspspike @mrbbot

[mrbbot]

Hey @paulrostorp! Thanks for the ping. I'll follow up with the team and see if we can get this landed. We're currently restricting which changes go into Wrangler due to some recent incidents. 👍

[lrapoport-cf]

I've got a PR for this opened, it seems to have gone unnoticed.... #4475

@caass @jspspike @mrbbot

hi @paulrostorp :) thanks for the contribution! following up on @mrbbot 's comment, we’re finishing the year focusing on work related to stability and bug fixes. given the scope of the PR, we won't be merging it now but will add the blocked-CO label to it and evaluate it with other enhancement work in early 2024 👍

[samifouad]

for anyone coming from a search engine who's interested in a temporary workaround, I'm using nginx as a reverse proxy until the PR merges

server {
    listen 443 ssl;
    server_name you.can.give.your.worker.a.local.hostname;

    ssl_certificate /path/to/cert.crt;
    ssl_certificate_key /path/to/key.key;

    location / {
        proxy_pass http://localhost:9123;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

if you have multiple workers and/or want to pin which port to configure inside nginx, change your package.json file start script:

"start": "wrangler dev --port 9123"

[lrapoport-cf]

hi @paulrostorp , i've removed the blocked-CO label 👍 we'll work on getting the PR reviewed, thanks!