-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add QuickJS as a Javascript engine option #4627
Conversation
looks really good. what would a NIF look like? I mentioned elsewhere that rustler would be a nice way to do that safely. I think you said 100% NIF is conditional on dropping _list/_show support? |
A NIF would be easier to do if we didn't have the getRow/sendRow sub-protocol for list/show, where we have to call back into Erlang, wait for a row, then return back to Javascript. As a driver it's just simpler to do with print and readline. Even then, it might still make sense to retain external driver vs NIF duality, as it is possible to apply more restrictive isolation policies (seccomp, apparmor, selinux, etc) to the OS process. With all the Rustler might improve the situation if there was a Rust-only JS engine which doesn't wrap a C or C++ one. Then, we could have a NIF and feel like we're not making as much of a safety vs speed tradeoff. I've been keeping an eye on https://github.com/Starlight-JS/Starlight and https://github.com/boa-dev/boa. Boa one looks more promising. |
fair enough. |
30ff2f7
to
9d255ea
Compare
8386088
to
331c8eb
Compare
Some progress update:
TODO:
|
997c267
to
79e9f8a
Compare
441fca1
to
d55a944
Compare
Hey Nick, thats really great work. I installed the MSYS2 choco package to the Win-CI and was able to compile CouchDB with QuickJS on Windows. We need an additional small change (PR) in CouchDB Glazier that this works out-of-the-box.
Running the tests:
|
Thanks for checking, Ronny! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey Nick,
a few small remarks after a quick read. Great work so far!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just leaving a couple comments, but looking great so far!
2211e0a
to
cc87df1
Compare
626d9cc
to
0810831
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks awesome!
Sorry if it's in the code and I missed it, but my only question is whether users of the scanner plugin might want to be warned when they exceed various max_
limits?
Some benefits over Mozilla Spidermonkey: * Small. We're using 6 or so C files vs 700+ SM91 C++ files. * Built with Apache CouchDB as opposed having to maintain a separate SM package, like for RHEL9, for instance, where they dropped support for SM already [1]. * Embedding friendly. Designed from ground-up for embedding. SM has been updating the C++ API such that we have to keep copy-pasting new versions of our C++ code every year or so [2]. * Easy to modify to accept Spidermonkey 1.8.5 top level functions for map/reduce code so we don't have have to parse the JS, AST transform it, and then re-compile it. * Better sandboxing. Creating a whole JSRuntime takes only 300 microseconds, so we can afford to do that on reset. JSRuntimes cannot share JS data or object between them. * Seems to be faster in preliminary benchmarking with small concurrent VDU and view builds: https://gist.github.com/nickva/ed239651114794ebb138b1f16c5f6758 Results seem promising: - 4x faster than SM 1.8.5 - 5x faster than SM 91 - 6x reduced memory usage per couchjs process (5MB vs 30MB) * Allows compiling JS bytecode ahead of time a C array of bytes. QuickJS can be built alongside Spidermonkey and toggled on/off at runtime: ``` ./configure --dev --js-engine=quickjs ``` This makes it the default engine. But Spidermonkey can still be set in the config option. ``` [couchdb] js_engine = spidermonkey | quickjs ``` To test individual views, without switching the default use the `javascript_quickjs` language in the design docs. To keep using Spidermonkey engine after switching the default, can use `javascript_spidermonkey` language in design docs. However, language selection will reset the view and the view will have to be rebuilt. It's also possible to build without Spidermonkey support completely by using: ``` ./configure --disable-spidermonkey ``` Issue: #4448 [1] #4154 [2] #4305
https://bellard.org/quickjs
Some benefits over Mozilla Spidermonkey:
Small. We're using 6 or so C files vs 700+ SM91 C++ files.
Built with Apache CouchDB as opposed having to maintain a separate SM package, like for RHEL9, for instance, where they dropped support for SM already: RHEL 9 Support for CouchDB #4154.
Embedding friendly. Designed from ground-up for embedding. SM has been updating the C++ API such that we have to keep copy-pasting new versions of our C++ code every year or so: try compiling for sm 102 #4305.
Easy to modify to accept Spidermonkey 1.8.5 top level functions for map/reduce code so we don't have have to parse the JS, AST transform it, and then re-compile it.
Better sandboxing. Creating a whole JSRuntime takes only 300 microseconds, so we can afford to do that on reset. JSRuntimes cannot share JS data or object between them.
Seems to be faster in preliminary benchmarking with small concurrent VDU and view builds:
https://gist.github.com/nickva/ed239651114794ebb138b1f16c5f6758
Allows compiling JS bytecode ahead of time a C array of bytes.
QuickJS can be built alongside Spidermonkey and toggled on/off at runtime:
This makes it the default engine. But Spidermonkey can still be set in the config option.
To test individual views, without switching the default use the
javascript_quickjs
language in the design docs. To keep using Spidermonkey engine after switching the default, can usejavascript_spidermonkey
language in design docs. However, language selection will reset the view and the view will have to be rebuilt.It's also possible to build without Spidermonkey support completely by using:
Issue: #4448
Some basic benchmarking
Summary:
Setup Details
MacOS Intel, Erlang 24, ./dev/run -n1
Benchmark script: https://gist.github.com/nickva/c0cbf6a556cc2dc7dd6ee79f504f5f84.
Results measured by the zsh
time
command as% time ./stampede_dbs_ddocs_vdus.py ...
. CPU usage is reported for the client not the couchjs process or Erlang VM. Some example of resource usage is provided below as btop screenshots.QuickJS
Spidermonkey 1.8.5
Spidermonkey 91
Using btop locally with one Spidermonkey run and two QuickJS runs:
Memory usage 30-32MB RSS for Spidermonkey and 4-5MB for QuickJS. The time it took to run the benchmark can seen in the CPU usage graph.