[BUG] RapidsShuffleHeartbeatManager needs to remove executors that are stale

[abellina]

We only add executors in RapidsShuffleHeartbeatManager but we never remove them.

We need to keep track of the last time we saw a heartbeat, and as we get more registrations check if any executors are stale, and not include in executor update messages. We don't need to tell already registered executors about the loss of a peer, because UCX should be able to figure that out on its own.

We also need to update the logic on how the "new executors" for a peer is decided. Currently it's based on registration order, and it will need to be time based I believe.

[abellina]

This PR #2587 is related and improves things slightly for 21.06.