Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to locate bindings or generate keys with Wasm backend. (node:4201) #3

Closed
StringManolo opened this issue Jul 9, 2024 · 5 comments

Comments

@StringManolo
Copy link

My enviroment is Aarch64 Alpine Linux (musl)

Here is the warning:

(node:4577) Warning: Using WebAssembly backend: Could not locate the bindings file. Tried:
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/Debug/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/Release/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/out/Debug/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/Debug/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/out/Release/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/Release/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/default/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/compiled/20.13.1/linux/arm64/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/addon-build/release/install-root/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/addon-build/debug/install-root/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/addon-build/default/install-root/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/lib/binding/node-v115-linux-arm64/node_pqclean.node

The warning triggers when calling:

await PQClean.kem.generateKeyPair('kyber1024');

The generateKeyPair method is unable to generate the keys:

{"publicKey":{},"privateKey":{}}
@StringManolo
Copy link
Author

StringManolo commented Jul 9, 2024

Sorry my mistake. I Didn't used properly the export method.

Here is the right way in case someone end's up here with same issue:

const { publicKey, privateKey } = await PQClean.kem.generateKeyPair('kyber1024');
const publicKeyHex = Buffer.from(publicKey.export()).toString('hex');
const privateKeyHex = Buffer.from(privateKey.export()).toString('hex');
  • Also to avoid the warning in node, i use next line before the imports of the libs:
process.removeAllListeners('warning');

If anyone needs an example of a full implementation of encryption/decryption:
https://github.com/StringManolo/DarkMessengerCrypto/blob/6d6f63d53c49ae9505c85efa9c70255879f649e3/crypto_modules/KYBER_module.js

(Usually you do 2 way comunication for the shared secret. This module is for 1 way comunication, but you can tune it)

@tniessen
Copy link
Owner

tniessen commented Jul 9, 2024

@StringManolo Glad to hear it's working! Are you intentionally avoiding the native binding (and thus using the WebAssembly backend) or is there an issue that's causing your application to use the WebAssembly backend (and thus emit the warning)?

@StringManolo
Copy link
Author

@tniessen i'm using the default config (pref-native):

npm run build-native

> pqclean@0.4.0 build-native
> node-gyp rebuild -j max

gyp info it worked if it ends with ok
gyp info using node-gyp@10.1.0
gyp info using node@20.13.1 | linux | arm64
gyp info find Python using Python version 3.12.3 found at "/usr/bin/python3"

gyp http GET https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz
gyp http fetch GET https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz attempt 1 failed with ETIMEDOUT
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz failed, reason:
gyp ERR! stack at ClientRequest.<anonymous> (/usr/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:130:14)
gyp ERR! stack at ClientRequest.emit (node:events:519:28)
gyp ERR! stack at _destroy (node:_http_client:880:13)
gyp ERR! stack at onSocketNT (node:_http_client:900:5)
gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
gyp ERR! System Linux 6.2.1-PRoot-Distro
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "-j" "max"
gyp ERR! cwd /projects/DarkMessengerCrypto/node_modules/pqclean
gyp ERR! node -v v20.13.1
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok

Is the download itself what is failling rigth ? Curl seems to be able to reach:

curl https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz -v
*   Trying 104.20.23.46:443...
*   Trying 2606:4700:10::6814:172e:443...
* Immediate connect fail for 2606:4700:10::6814:172e: Host is unreachable
*   Trying 2606:4700:10::6814:162e:443...
* Immediate connect fail for 2606:4700:10::6814:162e: Host is unreachable
* Connected to nodejs.org (104.20.23.46) port 443 (#0)
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=*.nodejs.org
*  start date: Feb 28 00:00:00 2024 GMT
*  expire date: Mar 30 23:59:59 2025 GMT
*  subjectAltName: host "nodejs.org" matched cert\'s "nodejs.org"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
> GET /download/release/v20.13.1/node-v20.13.1-headers.tar.gz HTTP/1.1
> Host: nodejs.org
> User-Agent: curl/7.82.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 10 Jul 2024 08:21:07 GMT
< Content-Type: application/gzip
< Content-Length: 8760725
< Connection: keep-alive
< last-modified: Thu, 09 May 2024 09:31:47 GMT
< etag: "663c9803-85ad95"
< x-robots-tag: noindex
< CF-Cache-Status: HIT
< Accept-Ranges: bytes
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Content-Type-Options: nosniff
< Server: cloudflare
< CF-RAY: 8a0f23705e4d6669-MAD
<
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):

File manually downloaded with curl:

$ du node-v20.13.1-headers.tar.gz -h
8.4M    node-v20.13.1-headers.tar.gz

$ md5sum node-v20.13.1-headers.tar.gz
5ac3eb8eb997be14965825e5d1e5b60e  node-v20.13.1-headers.tar.gz

Line 130 at /usr/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js is:

req.on('error', er => {
      // if a 'response' event is emitted before the 'error' event, then by the
      // time this handler is run it's too late to reject the Promise for the
      // response. instead, we forward the error event to the response stream
      // so that the error will surface to the user when they try to consume
      // the body. this is done as a side effect of aborting the request except
      // for in windows, where we must forward the event manually, otherwise
      // there is no longer a ref'd socket attached to the request and the
      // stream never ends so the event loop runs out of work and the process
      // exits without warning.
      // coverage skipped here due to the difficulty in testing
      // istanbul ignore next
      if (req.res) {
        req.res.emit('error', er)
      }
      reject(new FetchError(`request to ${request.url} failed, reason: ${
        er.message}`, 'system', er))
      finalize()
    })

@tniessen
Copy link
Owner

@StringManolo Thanks for looking into it. Unfortunately, these are not elements that I control.

The error message indicates a timeout, which might be related to node's odd Happy Eyeballs implementation (see, for example, nodejs/node#52216 or https://github.com/orgs/nodejs/discussions/48028). Assuming this is the issue, as a workaround, could you try setting the environment variable NODE_OPTIONS=--no-enable-network-family-autoselection for the build process, and see if this resolves the download issue?

(Alternatively, maybe a more recent version of Node.js is less buggy.)

@StringManolo
Copy link
Author

@tniessen Ty very much. Is built now and working.

The export NODE_OPTIONS=--no-enable-network-family-autoselection did the trick for downloading the file.

Also, after running npm run build-native got this error:

CC(target) Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/crypto_int16.o
CC(target) Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/crypto_int32.o
In file included from
../deps/PQClean/crypto_kem/mceliece348864/clean/controlbits.o] Error 1
../deps/PQClean/common/compat.h:20:21: error:
missing binary operator before token "("
20 | #  if !__GNUC_PREREQ(7, 1) // at least GCC 7.1
|                     ^
CC(target) Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/crypto_uint16.o
make: *** [native/gen/pqclean_kem_mceliece348864.target.mk:121:
Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/controlbits.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/projects/DarkMessengerCrypto/node_modules/pqclean/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:209:23)
gyp ERR! System Linux 6.2.1-PRoot-Distro
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "-j" "max"
gyp ERR! cwd /projects/DarkMessengerCrypto/node_modules/pqclean
gyp ERR! node -v v20.13.1
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok

Fixed it by editing the deps/PQClean/common/compat.h and adding the missing definition:

#ifndef __GNUC_PREREQ
  #define __GNUC_PREREQ(maj, min) ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min))
#endif

Added it just before this definition:

#if defined(__GNUC__) && !defined(__clang__)

I'm using this version of gcc:

gcc (Alpine 13.2.1_git20240309) 13.2.1 20240309
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants