Possibly add CORS middleware

[CasperWA]

Considering this issue Materials-Consortia/OPTIMADE#249 and PR Materials-Consortia/OPTIMADE#105 in the spec repo, it may be good to know that Starlette have a CORSMiddleware class.

Edit: FastAPI also has its own tutorial on this.

[ml-evs]

This looks like it would be easy enough to set up for us, would our whitelist include just the OPTiMaDe providers as discussed in the linked issue?

[CasperWA]

(...) would our whitelist include just the OPTiMaDe providers as discussed in the linked issue?

Don't know. I will let you take this issue, if you want. I just saw the discussion on the spec repo and added it here for reference :) By the way it seems the PR Materials-Consortia/OPTIMADE#140 is the one that implements it in the spec.

[blokhin]

You’re using a free Heroku account, right?

[ml-evs]

@ltalirz is the one to ask, but I assume so...

[blokhin]

@ltalirz please, advice (see #249)

[ltalirz]

yes, using a free heroku account.
this test server was mainly meant to always have a test instance running the current master branch of the python-optimade-tools.
in the same spirit, it can be used to test CORS - I would just not suggest to use it as the "central relay server" (both because it's a free account and heroku will shut it down after 30 minutes of inactivity; and because stuff might break in master at some point).

if such a server is in demand, we can make another instance that is deployed independently with sufficient resources.
We can happily host a first version of this on our server resources at CSCS (right, @giovannipizzi ?) - if traffic at some point becomes huge (let's hope it does!), we can look further .

Let me know - I'd then simply ask @gmrigna for a redirect of something like api.optimade.org to our server.

P.S. If people want this server on "neutral ground", we can also collect some money together for hosting it on heroku, where multiple people can easily have access to deployment and stats.

[blokhin]

Thanks for clarification! I have deployed the CORS-bypassing proxy at the Heroku, one can see how it works:

curl -v -H Origin:example.com http://odbx.science/optimade/

Result: no CORS header

curl -v -H Origin:example.com https://optimade-cors.herokuapp.com/http://odbx.science/optimade/

Result: CORS header is present: Access-Control-Allow-Origin: *

@ltalirz given we deploy it from the Optimade org github, could we reserve an org sub-domain for it? e.g. cors.optimade.org (SSL cert is needed).

PS
provider white-listing (preferably, dynamic one with fetching from the repo), as well as dealing with the 30-minutes limit are coming soon.

[ltalirz]

Nice!

could we reserve an org sub-domain for it? e.g. cors.optimade.org (SSL cert is needed).

I'm sure that is possible - as mentioned we should just ask @gmrigna to add a corresponding DNS entry - in this case it would be:
point cors.optimade.org to optimade-cors.herokudns.com - and then we use heroku domains:add cors.optimade.org, which will also set up the SSL cert via let's encrypt (see heroku instructions).

dealing with the 30-minutes limit

hm... should we really be doing that? For testing a free account is fine but if we really want to advertise this URL publicly, I guess we can afford to pay (or just use server resources from some of the many optimade partners)

[blokhin]

Overcoming the 30-minutes Heroku limit is not a problem at all. Any monitoring solutions (there are plenty of them, free) simply ping the app regularly. Obviously, Heroku understands that very well, but it's a kind of grey zone, they can neither prohibit it, nor encourage.

[blokhin]

Okay, Heroku wants $84 / year for a possibility to use a free SSL cert. Bye-bye Heroku (and its 30-minutes limit) and servus Zeit! It supports gh, custom domains, and gives you the SSL cert, all for free. The free limits are also quite tasty. I transferred a CORS-bypassing server repo under the Optimade gh org, and deployed it at the Zeit platform from there at https://cors.optimade.science.

@gmrigna could we use https://cors.optimade.org for that? It's enough to add just two DNS records, CNAME and TXT.

[blokhin]

UPD: Zeit does not support :// in paths (which is correct), so the web-browser clients need to replace :// with / in their xhr / fetch requests. I modified the server to account that.

[gmrigna]

I have asked our system administrator a few days ago... and I am still waiting for the answer (I will send a reminder).

[blokhin]

Great!

https://cors.optimade.org (source) is now fully operational.

[CasperWA]

This issue is not whether or not to add CORS to heroku or providers.optimade.org, but rather if it should be added by default to the the server implementation in this repository.

[blokhin]

This CORS proxy is now heavily used by the Sky-Scanner web-app https://optimade.science