Standardize how we log and handle panics

[RebeccaMahany]

Requires #1693

If one goroutine panics, all of launcher will terminate. Before termination, the panic will travel up to the top-level function in the executing goroutine. If we want to recover or log panics, we must do that in this top-level function.

This PR introduces gowrapper to wrap our goroutines, to make it easier to standardize recovering from and logging panics. It will always log the panic, and allows for providing an onPanic function that executes if gowrapper recovers from a panic.

This PR adds gowrapper usage to our windows service and to all of our rungroups.

References:

• https://forum.golangbridge.org/t/why-does-go-terminate-the-whole-process-if-one-goroutine-paincs/27122

Example logs from panic inside a rungroup

{
  "time":"2024-04-30T14:17:34.976926Z",
  "level":"ERROR",
  "source":{
    "function":"github.com/kolide/launcher/ee/gowrapper.Go.func1.1",
    "file":"/Users/rebeccamahany-horton/Repos/launcher/ee/gowrapper/goroutine.go",
    "line":18
  },
  "msg":"panic occurred in goroutine",
  "launcher_run_id":"01HWQNTJJ3RET2810PM254GV7H",
  "component":"run_group",
  "err":"runtime error: index out of range [3] with length 0"
}
{
  "time":"2024-04-30T14:17:34.983632Z",
  "level":"ERROR",
  "source":{
    "function":"github.com/kolide/launcher/ee/gowrapper.Go.func1.1",
    "file":"/Users/rebeccamahany-horton/Repos/launcher/ee/gowrapper/goroutine.go",
    "line":23
  },
  "msg":"panic stack trace",
  "launcher_run_id":"01HWQNTJJ3RET2810PM254GV7H",
  "component":"run_group",
  "stack_trace":"runtime error: index out of range [3] with length 0\ngitpro.ttaallkk.top/kolide/launcher/ee/gowrapper.Go.func1.1\n\t/Users/rebeccamahany-horton/Repos/launcher/ee/gowrapper/goroutine.go:25\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:914\nruntime.goPanicIndex\n\t/usr/local/go/src/runtime/panic.go:114\ngitpro.ttaallkk.top/kolide/launcher/ee/tuf.(*TufAutoupdater).Execute\n\t/Users/rebeccamahany-horton/Repos/launcher/ee/tuf/autoupdate.go:234\ngitpro.ttaallkk.top/kolide/launcher/pkg/rungroup.(*Group).Run.func1\n\t/Users/rebeccamahany-horton/Repos/launcher/pkg/rungroup/rungroup.go:71\ngitpro.ttaallkk.top/kolide/launcher/ee/gowrapper.Go.func1\n\t/Users/rebeccamahany-horton/Repos/launcher/ee/gowrapper/goroutine.go:33\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_arm64.s:1197"
}

Example logs from panic inside Windows service

{
  "time":"2024-04-30T14:26:47.1429033Z",
  "level":"ERROR",
  "source":{
  "function":
    "github.com/kolide/launcher/ee/gowrapper.Go.func1.1",
    "file":"C:/Users/Becca/Repos/launcher/ee/gowrapper/goroutine.go",
    "line":18
  },
  "msg":"panic occurred in goroutine",
  "launcher_run_id":"01HWQPD75EQTGQFQ4BBZ1FNFGD",
  "err":"runtime error: index out of range [1] with length 0"
}
{
  "time":"2024-04-30T14:26:47.1434689Z",
  "level":"ERROR",
  "source":{
    "function":"github.com/kolide/launcher/ee/gowrapper.Go.func1.1",
    "file":"C:/Users/Becca/Repos/launcher/ee/gowrapper/goroutine.go",
    "line":23
  },
  "msg":"panic stack trace",
  "launcher_run_id":"01HWQPD75EQTGQFQ4BBZ1FNFGD",
  "stack_trace":"runtime error: index out of range [1] with length 0\ngitpro.ttaallkk.top/kolide/launcher/ee/gowrapper.Go.func1.1\n\tC:/Users/Becca/Repos/launcher/ee/gowrapper/goroutine.go:25\nruntime.gopanic\n\tC:/Program Files/Go/src/runtime/panic.go:914\nruntime.goPanicIndex\n\tC:/Program Files/Go/src/runtime/panic.go:114\nmain.runLauncher\n\tC:/Users/Becca/Repos/launcher/cmd/launcher/launcher.go:525\nmain.(*winSvc).Execute.func1\n\tC:/Users/Becca/Repos/launcher/cmd/launcher/svc_windows.go:189\ngitpro.ttaallkk.top/kolide/launcher/ee/gowrapper.Go.func1\n\tC:/Users/Becca/Repos/launcher/ee/gowrapper/goroutine.go:33\nruntime.goexit\n\tC:/Program Files/Go/src/runtime/asm_amd64.s:1650"
}
{
  "time":"2024-04-30T14:26:47.1434689Z",
  "level":"ERROR",
  "source":{
    "function":"main.(*winSvc).Execute.func2",
    "file":"C:/Users/Becca/Repos/launcher/cmd/launcher/svc_windows.go",
    "line":205
  },
  "msg":"exiting after runLauncher panic",
  "launcher_run_id":"01HWQPD75EQTGQFQ4BBZ1FNFGD",
  "err":"runtime error: index out of range [1] with length 0"
}

[James-Pickett]

LGTM!

[zackattack01]

🔥