Embedded nats-server shutdown panics with: panic: close of nil channel

[sequitor-jon]

Observed behavior

Running this Go program and pressing Ctrl+C demonstrates the issue:

package main

import (
    "context"
    "errors"
    "log"
    "os"
    "syscall"
    "time"

    "github.com/nats-io/nats-server/v2/server"
    "github.com/oklog/run"
)

var errNotReady = errors.New("server not ready for connections")

func main() {
    opts := &server.Options{
        Port: 4222,
    }

    ns, err := server.NewServer(opts)
    if err != nil {
        log.Fatal(err)
    }

    var group run.Group

    group.Add(func() error {
        ns.Start()
        if !ns.ReadyForConnections(5 * time.Second) {
            return errNotReady
        }

        log.Println("server ready for connections")
        ns.WaitForShutdown()
        log.Println("server shutdown")
        return nil
    },
        func(err error) {
            if !errors.Is(err, errNotReady) {
                log.Printf("shutting down: %v\n", err)
                ns.Shutdown()
                ns.WaitForShutdown()
            }
        })

    group.Add(run.SignalHandler(context.Background(), os.Interrupt, syscall.SIGTERM))

    if err := group.Run(); err != nil {
        log.Fatal(err)
    }

    log.Println("done")
}

The output from running and pressing Ctrl+C:

% go run main.go
2024/09/24 14:12:32 server ready for connections
^C2024/09/24 14:12:33 shutting down: received signal interrupt
panic: close of nil channel

goroutine 1 [running]:
github.com/nats-io/nats-server/v2/server.(*Server).shutdownEventing(0x1400016fb08)
	/Users/jon/go/pkg/mod/github.com/nats-io/nats-server/v2@v2.10.20/server/events.go:1734 +0x12c
github.com/nats-io/nats-server/v2/server.(*Server).Shutdown(0x1400016fb08)
	/Users/jon/go/pkg/mod/github.com/nats-io/nats-server/v2@v2.10.20/server/server.go:2430 +0x3c
main.main.func2({0x10172bac0, 0x140001a0300})
	/Users/jon/code/go/nats-server/main.go:43 +0xbc
github.com/oklog/run.(*Group).Run(0x1400023dec8)
	/Users/jon/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:47 +0x174
main.main()
	/Users/jon/code/go/nats-server/main.go:50 +0x278
exit status 2

Expected behavior

Shutdown called from another go routine should not panic.

Server and client version

Go.mod file:

module foo.tech/nats-server

go 1.23.1

require (
    github.com/nats-io/nats-server/v2 v2.10.20
    github.com/oklog/run v1.1.0
)

require (
    github.com/klauspost/compress v1.17.9 // indirect
    github.com/minio/highwayhash v1.0.3 // indirect
    github.com/nats-io/jwt/v2 v2.5.8 // indirect
    github.com/nats-io/nkeys v0.4.7 // indirect
    github.com/nats-io/nuid v1.0.1 // indirect
    golang.org/x/crypto v0.26.0 // indirect
    golang.org/x/sys v0.24.0 // indirect
    golang.org/x/time v0.6.0 // indirect
)

Host environment

Behavior observed on both macOS 15.0 and Ubuntu 22.04

Steps to reproduce

See main.go and go.mod file.

[ripienaar]

We should handle Shutdown() being called twice better, but what's happening here is your app has a signals handler and the server installed its own so the term / interrupt is handled twice, each calling Shutdown()

You can avoid this by setting NoSigs: true in your server opts

[sequitor-jon]

Ah, thanks! I suspected something like that.