Data Race between object.FromEvent() and (*Incident).RetriggerEscalations

[oxzi]

WARNING: DATA RACE
Write at 0x00c0007f3d00 by goroutine 72:
  github.com/icinga/icinga-notifications/internal/object.FromEvent()
      /go/src/github.com/Icinga/icinga-notifications/internal/object/object.go:140 +0x22ce
  github.com/icinga/icinga-notifications/internal/incident.ProcessEvent()
      /go/src/github.com/Icinga/icinga-notifications/internal/incident/incidents.go:218 +0x1a8
  github.com/icinga/icinga-notifications/internal/icinga2.(*Launcher).launch.func1()
      /go/src/github.com/Icinga/icinga-notifications/internal/icinga2/launcher.go:132 +0x335
  github.com/icinga/icinga-notifications/internal/icinga2.(*Client).worker()
      /go/src/github.com/Icinga/icinga-notifications/internal/icinga2/client.go:560 +0xaf5
  github.com/icinga/icinga-notifications/internal/icinga2.(*Client).Process.gowrap1()
      /go/src/github.com/Icinga/icinga-notifications/internal/icinga2/client.go:586 +0x38

Previous read at 0x00c0007f3d00 by goroutine 195174:
  github.com/icinga/icinga-notifications/internal/incident.(*Incident).RetriggerEscalations.func1()
      /go/src/github.com/Icinga/icinga-notifications/internal/incident/incident.go:256 +0x148
  github.com/icinga/icinga-notifications/internal/utils.RunInTx()
      /go/src/github.com/Icinga/icinga-notifications/internal/utils/utils.go:46 +0x26b
  github.com/icinga/icinga-notifications/internal/incident.(*Incident).RetriggerEscalations()
      /go/src/github.com/Icinga/icinga-notifications/internal/incident/incident.go:255 +0xbe6
  github.com/icinga/icinga-notifications/internal/incident.(*Incident).evaluateEscalations.func1()
      /go/src/github.com/Icinga/icinga-notifications/internal/incident/incident.go:500 +0x564

Goroutine 72 (running) created at:
  github.com/icinga/icinga-notifications/internal/icinga2.(*Client).Process()
      /go/src/github.com/Icinga/icinga-notifications/internal/icinga2/client.go:586 +0x5cc
  github.com/icinga/icinga-notifications/internal/icinga2.(*Launcher).launch.gowrap1()
      /go/src/github.com/Icinga/icinga-notifications/internal/icinga2/launcher.go:149 +0x38

Goroutine 195174 (running) created at:
  time.goFunc()
      /usr/local/go/src/time/sleep.go:177 +0x54

[julianbrost]

Probably not just with (*Incident).RetriggerEscalations but pretty much everything accessing the Object struct. I think two regular process event requests for the same object could overlap in way showing a similar problem.

[julianbrost]

Note that I wouldn't necessarily try to specifically add locking to prevent this but maybe reconsider locking on a higher level. At first glance, locking on the object instead of the incident doesn't sound like a bad idea, so that for each object, at most one event is processed at once.

This has a slight overlap with #266, I wouldn't consider it the same problem and expect fixes for both in the same PR, but it should be at least be taken into account insofar to avoid making it more difficult to fix the other one with the changes for one.