Skip to content

Commit

Permalink
MOVED: Proactively reconfigure when we hit a MOVED response
Browse files Browse the repository at this point in the history
I'm not 100% sure about this because if there is a MOVED happening (e.g. bad proxy somewhere) this would just continually re-run...but only once every 5 seconds. Overall though, we linger in a bad state retrying moves until a discovery happens today and this could be resolved much faster.

Meant to help address #1520, #1660, #2074, and #2020.
  • Loading branch information
NickCraver committed Oct 27, 2022
1 parent e2e4c19 commit cf22311
Showing 1 changed file with 18 additions and 2 deletions.
20 changes: 18 additions & 2 deletions src/StackExchange.Redis/ConnectionMultiplexer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ public sealed partial class ConnectionMultiplexer : IInternalConnectionMultiplex

ConfigurationOptions IInternalConnectionMultiplexer.RawConfig => RawConfig;

private int lastReconfigiureTicks = Environment.TickCount;
internal long LastReconfigureSecondsAgo =>
unchecked(Environment.TickCount - Thread.VolatileRead(ref lastReconfigiureTicks)) / 1000;

private int _activeHeartbeatErrors, lastHeartbeatTicks;
internal long LastHeartbeatSecondsAgo =>
pulse is null
Expand Down Expand Up @@ -366,8 +370,19 @@ internal void CheckMessage(Message message)
}
}

internal bool TryResend(int hashSlot, Message message, EndPoint endpoint, bool isMoved) =>
ServerSelectionStrategy.TryResend(hashSlot, message, endpoint, isMoved);
internal bool TryResend(int hashSlot, Message message, EndPoint endpoint, bool isMoved)
{
// If we're being told to re-send something because the hash slot moved, that means our topology is out of date
// ...and we should re-evaluate what's what.
// Allow for a 5-second back-off so we don't hammer this in a loop though
if (isMoved && LastReconfigureSecondsAgo > 5)
{
// Async kickoff a reconfigure
ReconfigureIfNeeded(endpoint, false, "MOVED encountered");
}

return ServerSelectionStrategy.TryResend(hashSlot, message, endpoint, isMoved);
}

/// <summary>
/// Wait for a given asynchronous operation to complete (or timeout).
Expand Down Expand Up @@ -1214,6 +1229,7 @@ internal async Task<bool> ReconfigureAsync(bool first, bool reconfigureAll, LogP
}
Trace("Starting reconfiguration...");
Trace(blame != null, "Blaming: " + Format.ToString(blame));
Interlocked.Exchange(ref lastReconfigiureTicks, Environment.TickCount);

log?.WriteLine(RawConfig.ToString(includePassword: false));
log?.WriteLine();
Expand Down

0 comments on commit cf22311

Please sign in to comment.