Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calling async API from node worker threads results in errors #489

Closed
kdy1 opened this issue Feb 18, 2020 · 17 comments
Closed

Calling async API from node worker threads results in errors #489

kdy1 opened this issue Feb 18, 2020 · 17 comments

Comments

@kdy1
Copy link

kdy1 commented Feb 18, 2020

Related: swc-project/node-swc#21

Error message:

FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place

@dherman
Copy link
Collaborator

dherman commented Feb 29, 2020

@kdy1 Thank you for the report! Would you be able to provide us with a small code example so we can reproduce what you're trying to make work? Thank you!

@kdy1
Copy link
Author

kdy1 commented Feb 29, 2020

@dherman I've created a basic version of module which invokes native addon from worker thread.
See: https://github.com/kdy1/neon-worker-thread

@PSeitz
Copy link

PSeitz commented Sep 8, 2020

Worker support is documented here:
https://nodejs.org/api/addons.html#addons_worker_support

@Lyoko-Jeremie
Copy link

Lyoko-Jeremie commented Jan 17, 2021

@kdy1 seems like it not work now.

sleeping done
scheduled
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place

@kjvalencik
Copy link
Member

@Lyoko-Jeremie The N-API backend does not have this issue.

@Lyoko-Jeremie
Copy link

@kjvalencik where/when can i have it ?

@kjvalencik
Copy link
Member

@Lyoko-Jeremie
Copy link

@kjvalencik

same reasult in the worker_thread

sleeping: ThreadId(4)
sleeping done
scheduled
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place

i use the config


[build-dependencies]
neon-build = "0.7"

[dependencies]
#neon = { version = "0.6.0", features = ["event-handler-api"] }
neon = { version = "0.7", default-features = false, features = ["default-panic-hook", "napi-6", "try-catch-api", "event-queue-api"] }

code

pub fn async_task(mut cx: FunctionContext) -> JsResult<JsUndefined> {
    let undefined = cx.undefined();

    let cb = cx.argument::<JsFunction>(0)?.root(&mut cx);
    let queue = cx.queue();

    std::thread::spawn(move || {

        println!("sleeping: {:?}", thread::current().id());
        thread::sleep(std::time::Duration::from_secs(5));
        println!("sleeping done");
        queue.send(move |mut cx| {
            let callback = cb.into_inner(&mut cx);
            let this = cx.undefined();
            let null = cx.null();

            println!("Inside schedule fn");

            let args: Vec<Handle<JsValue>>  = vec![
                cx.string("abc").upcast::<JsValue>(),
                cx.number(100).upcast()
            ];

            callback.call(&mut cx, this, args)?;

            Ok(())
        });
        println!("scheduled");
    });

    Ok(undefined)
}


register_module!(mut cx, {
    cx.export_function("async_task", async_task)?;
    Ok(())
});

test it use :

// test.js
// https://github.com/neon-bindings/neon/issues/489
// https://github.com/kdy1/neon-worker-thread
const {
    Worker,
    isMainThread,
    parentPort,
    workerData
} = require("worker_threads");
const rxjs = require('rxjs');

if (isMainThread) {
    module.exports = function parseJSAsync(script) {
        const s=new rxjs.Subject();

        const worker = new Worker(__filename, {
            workerData: script
        });
        worker.on("message", s.next.bind(s));
        worker.on("error", s.error.bind(s));
        worker.on("exit", code => {
            if (code !== 0)
                s.error(new Error(`Worker stopped with exit code ${code}`));
            console.log(`Worker stopped with exit code ${code}`);
            s.complete();
        });
        return s;
    };
} else {
    const addon = require('../native');
    addon.async_task(function (err, v) {
        parentPort.postMessage("Error: ", err);
        parentPort.postMessage("Result: ", v);
    });
    setTimeout(() => {
        console.error('end')
    }, 1000 * 8)
}
// run.js
const t = require('./test')

console.log(t);
t().subscribe(
    value => {
        console.log(value)
    },
    error => {
        console.error(error)
    },
    () => {
        console.log('completed')
    }
);

@kjvalencik
Copy link
Member

kjvalencik commented Jan 19, 2021

@Lyoko-Jeremie Can you try deleting index.node and verifying the N-API version is being tested instead of a previous build?

I copied your code into a project and made minimal changes:

  • Added rxjs dependency
  • Added use statements to lib.rs so that the module compiles
  • Updated const addon = require('../native'); to load the module from the correct location

I ran the test 1000x and it passed successfully each time without the error.

$ node -v
v14.15.4

$ cargo version
cargo 1.51.0-nightly (329895f5b 2021-01-06)

$ rustc --version
rustc 1.51.0-nightly (a62a76047 2021-01-13)

@Lyoko-Jeremie
Copy link

node -v
v12.18.3


cargo version
cargo 1.49.0 (d00d64df9 2020-12-05)


rustc --version
rustc 1.49.0 (e1884a8e3 2020-12-29)

is it my node verion too low ?

i will try to upgrade it.

@Lyoko-Jeremie
Copy link

@kjvalencik

now i upgrade the nodejs


>node -v
v14.15.4

>cargo version
cargo 1.49.0 (d00d64df9 2020-12-05)

>rustc --version
rustc 1.49.0 (e1884a8e3 2020-12-29)

and a new problem happend

    Finished dev [unoptimized + debuginfo] target(s) in 35.35s
neon info generating native\index.node
neon ERR! ENOENT: no such file or directory, copyfile 'H:\Code\WebstormProjects\license-client\native\target\x86_64-pc-window
s-msvc\debug\license_client.dll' -> 'H:\Code\WebstormProjects\license-client\native\index.node'

Error: ENOENT: no such file or directory, copyfile 'H:\Code\WebstormProjects\license-client\native\target\x86_64-pc-windows-m
svc\debug\license_client.dll' -> 'H:\Code\WebstormProjects\license-client\native\index.node'

Process finished with exit code 1

i can see the license_client.dll.exp and license_client.dll.lib file are generated, but the license_client.dll not on it's place.

2021-01-19 12_41_44-Window


if i switch back to v0.6.0 like follow , all build will work well.


[build-dependencies]
neon-build = "0.6.0"
#neon-build = "0.7"

[dependencies]
neon = { version = "0.6.0", features = ["event-handler-api"] }
#neon = { version = "0.7", default-features = false, features = ["default-panic-hook", "napi-6", "try-catch-api", "event-queue-api"] }

where am i wrong ?

@Lyoko-Jeremie
Copy link

Lyoko-Jeremie commented Jan 19, 2021

now i install nightly version, license_client.dll still not on it's place.

> cargo version
cargo 1.51.0-nightly (a73e5b7d5 2021-01-12)

> rustc --version
rustc 1.51.0-nightly (d98d2f57d 2021-01-18)

@kjvalencik


BTW: if i remove the default-features = false, it cannot be compile.

@Lyoko-Jeremie
Copy link

Lyoko-Jeremie commented Jan 19, 2021

oh, i use the #662 fixed the dll issue.

now the new napi-6 is work.

@Lyoko-Jeremie
Copy link

now i get a Error: internal error in Neon module: not implemented error when use the Async Task feature

is it not impl now ? or some feature flag not enable ? or it will be delete in future ? @kjvalencik

@kjvalencik
Copy link
Member

Task will not be implemented in the N-API backend. It is completely removed in the next unreleased version.

Instead, use your own thread pool and callback in with EventQueue.

@Lyoko-Jeremie
Copy link

@kjvalencik thank you ~

@kjvalencik
Copy link
Member

Closing since this does not apply to the Node-API backend. Also, I found out I was wrong on the usefulness of node worker pool tasks and they will be implemented. neon-bindings/rfcs#35

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants