Sitecore – Configuring retries (defers) for the message bus

Sitecore has abstractions built in to make use of the message bus functionality which is implemented with Rebus. There’s a great post from Volodymyr how to use this functionality within your own Sitecore solution.

I wanted to make sure my messages were retried several times, as I’m calling an external API which isn’t available from time to time. You can use the Rebus way of doing things by using the Second-level retries, however this will introduce a direct dependency on Rebus and it’s bypassing the abstractions that Sitecore already created.

As an alternative, we can use a DeferStrategy in combination with a DeferDetectionPredicate and a DeferalScheduler, which are all defined in Sitecore.Framework.Messaging.Abstractions.dll.

Defer Detection Predicates

The defer detection predicates define if a message should be retried, the existing types are explained below.

AlwaysDeferOnError

As the name indicaties, this will always retry the message if an Exception is thrown. There are two variants, one that will defer on any type of exception and one where you can explicitly define the type of the exception.

DeferDetectionByResultBase

An abstract strategy which checks if the return type of the executed work is as configured and gives you the option to also inspect the result of the work. You’ll have to create a derived type of this class, below is an example from EXM.

  public class ExmDeferDetection : DeferDetectionByResultBase<HandlerResult>
    {
        public ExmDeferDetection()
        {
        }

        protected override bool ShouldDefer<TMessage>(TMessage message, HandlerResult result)
        {
            HandlerResultType handlerResultType = result.HandlerResultType;
            if (handlerResultType == HandlerResultType.Successful)
            {
                return false;
            }
            if (handlerResultType != HandlerResultType.Error)
            {
                throw new InvalidOperationException("Unknown defer response");
            }
            return true;
        }

        protected override bool ShouldDefer<TMessage>(TMessage message, Exception exception)
        {
            return exception is EmailCampaignException;
        }
    }

DeferDetectionByMessageBase

An abstract strategy which checks if the message is as configured and gives you the option to also inspect the message and result of the work. Below is an example.

    public class MyDeferStrategy : DeferDetectionByMessageBase<MyMessage>
    {
        protected override bool ShouldDefer<TResult>(MyMessage messageHandler, TResult result)
        {
            throw new NotImplementedException();
        }

        protected override bool ShouldDefer(MyMessage messageHandler, Exception exception)
        {
            throw new NotImplementedException();
        }
    }

NeverDefer

A strategy that never retries a message…

DeferalScheduler

Where the Defer Detection Predicates specifies if a message has to be retried, the DeferalScheduler defines when (in which time) a message should be retried. For example, after 10 minutes or exponential. There are a few default implementations which are explained below.

ExponentialDeferalScheduler

This scheduler will return a longer delay every time the message is retried. For example, starting with 10 minutes and eventually returns a delay of one day. It has the following parameters that need to be configured:

MaxDeferalAttempts
Specifies how many times you want to retry the message.

MinBackOff
The minimum delay after which a message can be retried.

MaxBackOff
The maximum delay after which a message can be retried.

DeltaBackOff
The value that will be used to calculate a random delta in the exponential deferral between message handling attempts.

Example
As an example, i’ve configured this strategy to the following settings:
MaxDeferalAttempts: 5
MinBackOff: 5 minutes
MaxBackOff: 24 hours
DeltaBackOff: 60 minutes

This will lead to the following delays:
d:hh:mm:ss
0:00:53:39
0:02:30:59
0:05:45:39
0:12:14:58
1:00:00:00,0000000

FixedDeferalScheduler

The fixed defer scheduler always returns the configured timespan as a delay. This scheduler has the following parameters:

MaxDeferalAttempts
Specifies how many times you want to retry the message.

DeferPeriod
The delay after which the message should be retried again.

IncrementalDeferalScheduler

The Incremental scheduler will return a linear delay every time a message is processed. This scheduler has the following parameters:

MaxDeferalAttempts
Specifies how many times you want to retry the message.

InitialDeferPeriod
The time period a message will be deferred after the first delivery attempt

DeferPeriodIncrement
The amount by which the defer period is increased each time a message is deferred

NoDeferalScheduler

Immediatly tries the message again

DeferStrategy

Now we’ve talked about the DeferDetectionPredicate and the DeferScheduler, it’s time to look at the DeferStrategy. The DeferStrategy is the class that does all of the work and is configured with the DeferDetectionPredicate and the DeferScheduler.

This class should be used within your MessageHandler, to be able to retry a message. There are several ways you can create a DeferStrategy:

  • Manually creating the DeferStrategy
  • Using the Sitecore Factory with Configuration

The examples below indicate how the DeferStrategies are created, but do not take Dependency Injection into account but just shows you how to create the objects. You can easily do this with dependency injection as well.

Manually creating the DeferStrategy

In this example I’m creating a strategy with AlwaysDeferOnError and ExponentalDeferalScheduler

        var predicate = new AlwaysDeferOnError();
        var scheduler = new ExponentialDeferalScheduler(5, TimeSpan.FromMinutes(5), TimeSpan.FromDays(1), TimeSpan.FromHours(1));
        var strategy = new DeferStrategy(predicate, scheduler);

Using the Sitecore Factory with Configuration

In this example I’m creating a strategy with AlwaysDeferOnError and ExponentalDeferalScheduler

        var strategy = Factory.CreateObject("YourConfigurationNode/MyDeferStategy", true) as DeferStrategy;
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/"
               xmlns:role="http://www.sitecore.net/xmlconfig/role"
               xmlns:messagingTransport="http://www.sitecore.net/xmlconfig/messagingTransport/">
  <sitecore>
    <YourConfigurationNode>
      <MyDeferStategy type="Sitecore.Framework.Messaging.DeferStrategies.DeferStrategy, Sitecore.Framework.Messaging.Abstractions" singleInstance="true">
        <param desc="detection" type="Sitecore.Framework.Messaging.DeferStrategies.AlwaysDeferOnError, Sitecore.Framework.Messaging.Abstractions" singleInstance="true"/>
        <param desc="scheduler" type="Sitecore.Framework.Messaging.DeferStrategies.ExponentialDeferalScheduler, Sitecore.Framework.Messaging.Abstractions" singleInstance="true">
          <param desc="maxDeferalAttempts" type="Sitecore.Marketing.Automation.ConfigurationHelper, Sitecore.Marketing.Automation" factoryMethod="ToInt32" arg0="5" />
          <param desc="minBackoff" type="Sitecore.Marketing.Automation.ConfigurationHelper, Sitecore.Marketing.Automation" factoryMethod="ToTimeSpan" arg0="00:05:00" />
          <param desc="maxBackoff" type="Sitecore.Marketing.Automation.ConfigurationHelper, Sitecore.Marketing.Automation" factoryMethod="ToTimeSpan" arg0="24:00:00" />
          <param desc="deltaBackoff" type="Sitecore.Marketing.Automation.ConfigurationHelper, Sitecore.Marketing.Automation" factoryMethod="ToTimeSpan" arg0="01:00:00" />
        </param>
      </MyDeferStategy>
    </YourConfigurationNode>
  </sitecore>
</configuration>

How to use the DeferStrategy and actually retrying your messages

The DeferStrategy should be available in your message, and you shoud explicitly call the ExecuteAsync method. Please see the example below.

    public class MyMessageHandler : IMessageHandler<MyMessage>
    {
        private readonly IMessageBus<MyMessageBus> _myMessageBus;

        public MyMessageHandler(IMessageBus<MyMessageBus> myMessageBus)
        {
            _myMessageBus = myMessageBus;
        }

        public async Task Handle(MyMessage message, IMessageReceiveContext receiveContext,
            IMessageReplyContext replyContext)
        {
            var strategy = GetDeferStrategy();
            try
            {
                //Execute the work and let the strategy decide if the message was processed correctly
                var deferResult = await strategy.ExecuteAsync(_myMessageBus, message, receiveContext, DoWork);
                if (deferResult.Deferred)
                {
                    //The DeferStrategy indicated that the message was not correctly processed
                }
                else
                {
                    //The DeferStrategy indicated that the message was correctly processed
                    //The result can be accessed trough the Result property and in this case contains the value of the method DoWork
                    var result = deferResult.Result;
                }
            }
            catch (DeferStrategyExceededException exception)
            {
                //The strategy has retried the message as many times as it's allowed to do.
                //If you do not catch the exception the message will be sent to the configured error queue.
            }
        }

        private object DoWork()
        {
            return null;
        }

        private DeferStrategy GetDeferStrategy()
        {
            var predicate = new AlwaysDeferOnError();
            var scheduler = new ExponentialDeferalScheduler(5, TimeSpan.FromMinutes(5), TimeSpan.FromDays(1),
                TimeSpan.FromHours(1));
            return new DeferStrategy(predicate, scheduler);
        }
    }

    public class MyMessage { }

    public class MyMessageBus { }

Leave a Reply

Your email address will not be published. Required fields are marked *