Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(parser/napi): add flexbuffer to AST transfer (2x speedup) #1680

Merged
merged 1 commit into from
Dec 15, 2023

Conversation

HerringtonDarkholme
Copy link
Contributor

@HerringtonDarkholme HerringtonDarkholme commented Dec 15, 2023

Hi! I have created a proof of concept of improving using oxc in JavaScript. The method is not polished but it provides valuable insights for future direction!

Feel free to close~ It is for reference only :)

Context

This is a proof of concept implementation of passing binary AST to JavaScript. JavaScript can selectively read flexbuffers-based AST nodes on demand to avoid the deserialization toll. More context here.

Changes

  • Add a parseSyncBuffer napi method to return a binary AST from Rust to JavaScript. The AST is in flexbuffer format.
  • Add a test_buffer.js to test usage of flexbuffers in JavaScript. It is in cjs format because flexbuffers does not support ESM :/

Result

Some preliminary results, for reference only.

~ node test_buffer.js
testJSON: 4.043s
testBuffer: 2.395s

Buffer based API is 100% faster than JSON.

Future Ideas

  • Flexbuffers itself is slow. A better binary protocol is desired!
  • Using binary reader to traverse AST is undesirable. A proxy-based API to emulate object behavior will be nice.

This is a proof of conecpt implementation of passing binary AST
to JavaScript. JavaScript can selectively read flexbuffers-based
AST nodes on demand to avoid serialization toll.
@Boshen Boshen changed the title feat(napi): add parseBuffer POC feat(parser/napi): add flexbuffer to transfer AST (2x speedup) Dec 15, 2023
@Boshen Boshen changed the title feat(parser/napi): add flexbuffer to transfer AST (2x speedup) feat(parser/napi): add flexbuffer to AST transfer (2x speedup) Dec 15, 2023
@Boshen Boshen enabled auto-merge (squash) December 15, 2023 02:49
@Boshen Boshen merged commit c63f512 into oxc-project:main Dec 15, 2023
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants