Random runtime crashes in pkg on Windows

[mikiher]

What version of pkg are you using?

5.12.0

What version of Node.js are you using?

18.5.0

What operating system are you using?

Windows

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node18-win-x64

Describe the Bug

This was reported by serveral audiobookshelf-windows users (I'm maintaining this app) in this bug.

In a nutshell, audiobookshelf-windows packs the Audiobookshelf server into a windows executable using yao-pkg and runs it from tray application. The pkg command and config can be found here. The server also uses a native addon (sqlite3) and pkg config puts the relevant .node files in the assets section.

Users have reported random crashes inside pkg runtime code, which I have not been able to reproduce, but I wanted to see if you've seen this before or have some advice.

The crashes seems to happen at random points, and all look roughly like this:

[2024-08-06 08:13:55.840] FATAL: [Server] Uncaught exception origin: uncaughtException, error: Error: ENOENT: no such file or directory, open 'C:\Users\shaun\AppData\Local\Temp\pkg-NGVozZ\88e6d330c6f684cce2af952a76696bfc3ba3d62a88e46fd9b599d7b54591a743'
    at Object.openSync (node:fs:599:3)
    at Object.openSync (pkg/prelude/bootstrap.js:793:32)
    at uncompressExternallyAndOpen (pkg/prelude/bootstrap.js:748:19)
    at pkg/prelude/bootstrap.js:763:32
    at FSReqCallback.oncomplete (node:fs:201:23) {
  errno: -4058,
  syscall: 'open',
  code: 'ENOENT',
  path: 'C:\\Users\\shaun\\AppData\\Local\\Temp\\pkg-NGVozZ\\88e6d330c6f684cce2af952a76696bfc3ba3d62a88e46fd9b599d7b54591a743'
} (    at process.<anonymous> (C:\snapshot\audiobookshelf\server\Server.js))
node:internal/fs/utils:347
    throw err;
    ^

Error: ENOENT: no such file or directory, open 'C:\Users\shaun\AppData\Local\Temp\pkg-NGVozZ\88e6d330c6f684cce2af952a76696bfc3ba3d62a88e46fd9b599d7b54591a743'
    at Object.openSync (node:fs:599:3)
    at Object.openSync (pkg/prelude/bootstrap.js:793:32)
    at uncompressExternallyAndOpen (pkg/prelude/bootstrap.js:748:19)
    at pkg/prelude/bootstrap.js:763:32
    at FSReqCallback.oncomplete (node:fs:201:23) {
  errno: -4058,
  syscall: 'open',
  code: 'ENOENT',
  path: 'C:\\Users\\shaun\\AppData\\Local\\Temp\\pkg-NGVozZ\\88e6d330c6f684cce2af952a76696bfc3ba3d62a88e46fd9b599d7b54591a743'
}

Expected Behavior

Obviously, not crash :)

To Reproduce

Unfortunately, I have not been able to repro this - I'm not quite sure how to trigger this as it seems to happen randomly and quite rarely.

[robertsLando]

Hi @mikiher , that C:\Users\shaun\AppData\Local\Temp\pkg-NGVozZ\ is where pkg copies .node and .dll files when them are required from the code, reason is that them cannot be loaded from snapshot like it happens for all other files. My feel is that for some reason that folder is automatically removed by Windows for some reason? Maybe the anti-virus detect that file as a virus and removes it from there? Unfortunately with the informations provided I cannot say much more.

The code that handles that is here:

https://github.com/yao-pkg/pkg/blob/main/prelude/bootstrap.js#L2199-L2255

[robertsLando]

The strange thing is the folder should be different. Starting from 5.12.0 we moved the cache folder to homedir: #55

Could you try to switch to pkg latest first?

I see you are not specifing a version here: https://github.com/mikiher/audiobookshelf-windows/blob/master/.github/workflows/build-installer.yml#L27

I suggest you to put a fixed version instead when installing it

[mikiher]

The strange thing is the folder should be different. Starting from 5.12.0 we moved the cache folder to homedir: #55

Ah, sorry for the confusion - some of the user reports may date back to when we used previous versions.

I suggest you to put a fixed version instead when installing it

OK, thanks for the suggestion, will do. It looks like I unintentionally used the latest version anyway when building the executable, but as I said above, some of the reports may have used older versions.

At which point in my code would dlopen() be called? if this has to do with loading the sqlite native addon, I expect this would happen at server startup, but it looks like the crashes occur at random times. Is the dll sometimes unloaded and needs to be reloaded?

[robertsLando]

At which point in my code would dlopen() be called

It is called when a dll is required in code somewhere (so in theory only on startup). Problem is that if for some reason sometihng (like an anti virus) or else cleans the dir where we store this files the error could happen also during runtime. I suggest you to try replicate the issue with latest version as we changed the location where we load those files and so the issue may not happen anymore