RedisCommandException for "Command cannot be issued to a replica" is difficult to handle

[AdamOutcalt]

Hey all,

I work on a large service using Azure Redis Premium. Our normal pattern for using Redis is something like:

try{
db.StringGet()
}
catch (RedisException)
...
catch (TimeoutException)
...

This works pretty well, but we have some low level of failures that look like this:

RedisCommandException: Exception: 'Command cannot be issued to a slave: UNLINK '

(In newer versions of the SDK, it says "Command cannot be issued to a replica" instead.)

This is a reasonable error, but I am not intentionally targeting a replica. I assume Azure has failed over from under me, changing the recipient to a replica, while it was primary when the command was sent.

I would like to just treat this as a transient failure, log it, and move on with life as though it were a timeout. This is a little tricky though because the exception is RedisCommandException (which doesn't inherit from RedisException). I am hesitant to catch and handle RedisCommandException, because I understand that it usually means the command is bad, like maybe a bad LUA script.

Basically, I'd like some way to differentiate this exception so I can treat it as transient.

My suggestion is to make this case a RedisException or RedisServerException, but I'm open to other resolutions, including being told that I should be catching some different set of exceptions as transient.

Thanks,
Adam

[NickCraver]

Which version of the library are you using currently?

[AdamOutcalt]

2.2.79 (currently shows "replica" in the text)

[hao9638422]

Same issue here:
StackExchange.Redis.RedisCommandException: Command cannot be issued to a replica: SETEX xxx

[pseidel-kcf]

Having this issue with UNLINK on AWS clustered Elasticache 6.x using StackExchange 2.6.45 (2.6.48 in our staging environment does not appear to resolve the issue). Explicitly adding DemandMaster as a parameter to KeyDeleteAsync mitigates the issue.

StackExchange.Redis.RedisCommandException: Command cannot be issued to a replica: UNLINK xxx
   at StackExchange.Redis.ConnectionMultiplexer.PrepareToPushMessageToBridge[T](Message message, ResultProcessor`1 processor, IResultBox`1 resultBox, ServerEndPoint& server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1742
   at StackExchange.Redis.ConnectionMultiplexer.TryPushMessageToBridgeAsync[T](Message message, ResultProcessor`1 processor, IResultBox`1 resultBox, ServerEndPoint& server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1788
   at StackExchange.Redis.ConnectionMultiplexer.ExecuteAsyncImpl[T](Message message, ResultProcessor`1 processor, Object state, ServerEndPoint server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1952
   at StackExchange.Redis.RedisBase.ExecuteAsync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/RedisBase.cs:line 54
   at StackExchange.Redis.RedisDatabase.KeyDeleteAsync(RedisKey key, CommandFlags flags) in /_/src/StackExchange.Redis/RedisDatabase.cs:line 759

[slorello89]

@NickCraver

Looks like the pattern of checking features and passing in the server might be proving to be a bit problematic.

Looks like the issue is propagating from here

When the feature check is done here, there's nothing that allows the writeability of the command to influence server-selection, so it's just a crap-shoot whether you get back a master or replica to pass further down the line.