Introduction
Retry mechanism is simple, well known and widely used in places where a call is possible to fail but more than one are more probable to succeed.
The advent of cloud computing and the incremental inclination to build applications and solutions based on the cloud and the need for designing robust and stable solution considering the instability of any network have put more emphasis on this retry mechanism.
Hence, there are a few related mechanisms that are documented in the shape of design patterns, I am talking mainly about the “Retry” and “Circuit Breaker” design patterns.
Using the Code
The code shows the seed building block only, although it is covered by tests and working, yet there is some wide room for enhancements. I will be working on that for my real production but this is some sample for other to reuse or build upon.
The code is used in the tests and also in the program.cs.
Just play with the parameters.
Try:
breakerToRecover = TimeSpan.FromSeconds(5);
then try:
breakerToRecover = TimeSpan.FromSeconds(2);
Also change these numbers and see different behaviors:
private const int numberOfFailures = 5;
private const int numberOfRetry = 5;
private const int numberOfCircuitTrials = 5;
The Code
The design is to support DI, Fluent interface, hence let's start with a simple interface that can be extended to support more functionalities.
public interface IRetryBlock
{
IRetryBlock On<ExceptionType>(Func<Exception, bool> condition = null);
IRetryBlock Retry(int retryCount, TimeSpan retryInterval);
Task<IRetryBlock> RetryAsync(int retryCount, TimeSpan retryInterval);
}
Then the implementation:
public abstract class RetryBlock : IRetryBlock
{
private Action _toDo;
private readonly ConcurrentDictionary<Type, Func<Exception, bool>> _transientExceptions;
private readonly Func<Exception, bool> defaultCondition = null;
internal RetryBlock(Action toDo)
{
_toDo += toDo;
_transientExceptions = new ConcurrentDictionary<Type, Func<Exception, bool>>();
}
protected virtual bool IsTransient(Exception ex)
{
if (ex is OperationTransientException) return true;
if (_transientExceptions.ContainsKey(ex.GetType()))
{
Func<Exception, bool> exceptionCondition;
_transientExceptions.TryGetValue(ex.GetType(), out exceptionCondition);
return exceptionCondition.Invoke(ex);
}
return false;
}
public virtual IRetryBlock On<ExceptionType>(Func<Exception, bool> condition = null)
{
_transientExceptions.TryAdd(typeof(ExceptionType), condition ?? defaultCondition);
return this;
}
public virtual IRetryBlock Retry(int retryCount, TimeSpan retryInterval)
{
var currentRetry = 0;
while (true)
{
try
{
this.Execute();
break;
}
catch (Exception ex)
{
currentRetry++;
if (currentRetry > retryCount || !IsTransient(ex))
{
throw;
}
}
Console.WriteLine($"Current Retry: {currentRetry}");
Thread.Sleep(retryInterval);
}
return this;
}
private IRetryBlock Execute()
{
if (_toDo == null) throw new NullReferenceException("_toDo");
_toDo?.Invoke();
return this;
}
public virtual Task<IRetryBlock> RetryAsync(int retryCount,
TimeSpan retryInterval) => Task.Run(() => { return Retry(retryCount, retryInterval); });
private Task<IRetryBlock> ExecuteAsync() => Task.Run(() => { return Execute(); });
}
Helper class is just to create an instance of the (IRetryBlock
), the configure method is for you to define an instance of another implementation.
public partial class Retry
{
private static Func<Action, IRetryBlock> _builder;
public static IRetryBlock Handle(Action toDo) => _builder == null ?
new DefaultRetryBlock(toDo) : _builder.Invoke(toDo);
public static void Configure(Func<Action, IRetryBlock> builder)
{
_builder = builder;
}
}
Points of Interest
There are some unit tests to cover some scenarios, I will add more in the complete version.
Circuit breaker is another design pattern that is also well known and used in the similar scenarios where you use the "Retry
", yet in my example, I used them together and tried multiple scenarios and showed how it acts.
Let’s see these examples:
private const int numberOfFailures = 5;
private const int numberOfRetry = 5;
private const int numberOfCircuitTrials = 5;
This will cause results like:
private const int numberOfFailures = 4;
private const int numberOfRetry = 5;
private const int numberOfCircuitTrials = 5;
private const int numberOfFailures = 5
private const int numberOfRetry = 5;
private const int numberOfCircuitTrials = 4;
and the like.
It all depends on how many failures are occurring before the first successful call.
Retry mechanism will take care of errors if the expected exception occurs less times than retry counts.
Circuit breaker will disconnect the system informing that system is unavailable if the number of failures reached the number configured in the circuit.
There is the timeout configuration if it is too small for the circuit to recover, it will succeed..
References
- Circuit Breaker Pattern
- Retry Pattern
- Coderview