Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core lightning crashes with FATAL SIGNAL 11 #7645

Closed
ShahanaFarooqui opened this issue Sep 6, 2024 · 6 comments · Fixed by #7664
Closed

Core lightning crashes with FATAL SIGNAL 11 #7645

ShahanaFarooqui opened this issue Sep 6, 2024 · 6 comments · Fixed by #7664
Assignees
Milestone

Comments

@ShahanaFarooqui
Copy link
Collaborator

Core lightning v24.08 is crashing with FATAL SIGNAL 11 if alias is missing.

References:
Crash-1
Crash-2

@cdecker
Copy link
Member

cdecker commented Sep 6, 2024

Looks like the culprit is *channel->alias[LOCAL]. Are we missing an alias at this time? We should be filling them in if they are missing, but maybe this is a channel loaded from the DB and predates aliases?

@surfac3
Copy link

surfac3 commented Sep 7, 2024

in case you would need some data or logs feel free to tell me what should i run and ill post in here.

I am the one who expirienced the issue. Some of those channels can be 2yrs old

I have tried:
reiniating backup - no change
removing gossip.store - no help
clean reflashing of fw of the raspiblitz - no help

I tried to run sudo journalctl -fu clrest if that can help
Sep 08 21:41:46 raspberrypi node[3123160]: at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
Sep 08 21:41:46 raspberrypi node[3123160]: errno: -111,
Sep 08 21:41:46 raspberrypi node[3123160]: code: 'ECONNREFUSED',
Sep 08 21:41:46 raspberrypi node[3123160]: syscall: 'connect',
Sep 08 21:41:46 raspberrypi node[3123160]: address: '/home/bitcoin/.lightning/bitcoin/lightning-rpc'
Sep 08 21:41:46 raspberrypi node[3123160]: }
Sep 08 21:41:46 raspberrypi node[3123160]: Node.js v20.5.1
Sep 08 21:41:46 raspberrypi systemd[1]: clrest.service: Main process exited, code=exited, status=1/FAILURE
Sep 08 21:41:46 raspberrypi systemd[1]: clrest.service: Failed with result 'exit-code'.
Sep 08 21:41:46 raspberrypi systemd[1]: clrest.service: Consumed 2.715s CPU time.
Sep 08 21:42:16 raspberrypi systemd[1]: clrest.service: Scheduled restart job, restart counter is at 10.
Sep 08 21:42:16 raspberrypi systemd[1]: Stopped clrest.service - c-lightning-REST daemon for mainnet.
Sep 08 21:42:16 raspberrypi systemd[1]: clrest.service: Consumed 2.715s CPU time.
Sep 08 21:42:16 raspberrypi systemd[1]: Started clrest.service - c-lightning-REST daemon for mainnet.
Sep 08 21:42:18 raspberrypi node[3126884]: node:events:492
Sep 08 21:42:18 raspberrypi node[3126884]: throw er; // Unhandled 'error' event
Sep 08 21:42:18 raspberrypi node[3126884]: ^
Sep 08 21:42:18 raspberrypi node[3126884]: Error: connect ECONNREFUSED /home/bitcoin/.lightning/bitcoin/lightning-rpc
Sep 08 21:42:18 raspberrypi node[3126884]: at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1595:16)
Sep 08 21:42:18 raspberrypi node[3126884]: Emitted 'error' event on LightningClient instance at:
Sep 08 21:42:18 raspberrypi node[3126884]: at Socket. (/home/bitcoin/c-lightning-REST/bitcoin/lightning-client-js.js:80:23)
Sep 08 21:42:18 raspberrypi node[3126884]: at Socket.emit (node:events:514:28)
Sep 08 21:42:18 raspberrypi node[3126884]: at emitErrorNT (node:internal/streams/destroy:151:8)
Sep 08 21:42:18 raspberrypi node[3126884]: at emitErrorCloseNT (node:internal/streams/destroy:116:3)
Sep 08 21:42:18 raspberrypi node[3126884]: at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
Sep 08 21:42:18 raspberrypi node[3126884]: errno: -111,
Sep 08 21:42:18 raspberrypi node[3126884]: code: 'ECONNREFUSED',
Sep 08 21:42:18 raspberrypi node[3126884]: syscall: 'connect',
Sep 08 21:42:18 raspberrypi node[3126884]: address: '/home/bitcoin/.lightning/bitcoin/lightning-rpc'
Sep 08 21:42:18 raspberrypi node[3126884]: }
Sep 08 21:42:18 raspberrypi node[3126884]: Node.js v20.5.1
Sep 08 21:42:18 raspberrypi systemd[1]: clrest.service: Main process exited, code=exited, status=1/FAILURE
Sep 08 21:42:18 raspberrypi systemd[1]: clrest.service: Failed with result 'exit-code'.
Sep 08 21:42:18 raspberrypi systemd[1]: clrest.service: Consumed 2.092s CPU time.

btw isnt this a very similar issue?

@surfac3
Copy link

surfac3 commented Sep 9, 2024

I tried to update to latest master - CLN v24.08-10-g78b9ccf - #7650

but it did not help

image

@rustyrussell
Copy link
Contributor

This is a different crash. We are somehow not setting the local alias, presumably a historical channel.

rustyrussell added a commit to rustyrussell/lightning that referenced this issue Sep 13, 2024
If we pull and old channel from the database, it might not.

Fixes: ElementsProject#7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit to rustyrussell/lightning that referenced this issue Sep 13, 2024
If we pull and old channel from the database, it might not.

Fixes: ElementsProject#7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit to rustyrussell/lightning that referenced this issue Sep 13, 2024
If we pull and old channel from the database, it might not.

Fixes: ElementsProject#7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this issue Sep 13, 2024
If we pull and old channel from the database, it might not.

Fixes: #7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
@whitslack
Copy link
Collaborator

@rustyrussell: How could this have happened if the database migration of b5bd907 had executed?

@surfac3
Copy link

surfac3 commented Sep 13, 2024

I have pulled the latest master and the fix works!

Succesfuly pulled CLN v24.08-13-g5bd3d51 and it purrs like a kitten

ShahanaFarooqui pushed a commit to ShahanaFarooqui/lightning that referenced this issue Sep 17, 2024
If we pull and old channel from the database, it might not.

Fixes: ElementsProject#7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants