Options for deadlock retry with ObjectFactory

Options for deadlock retry with ObjectFactory

Old forum URL: forums.lhotka.net/forums/t/8558.aspx


cds posted on Friday, February 19, 2010

I'm using the ObjectFactory approach for my data access for my business objects. In my application, there are odd occasions where I get deadlocks, and I'd like to be able to automatically abort the transaction and retry a number of times.

Has anybody implemented this, and is there an easy generic way I could implement this without having to visit every ObjectFactory class?

Cheers for any ideas...

Craig

RedShiftZ replied on Friday, February 19, 2010

I do this in my DataPortal Select itself.

private void DataPortal_Fetch()
    {
      RaiseListChangedEvents = false;
      using (var mgr = ConnectionManager<SqlConnection>.GetManager("MyDB"))
      {
        using (var cmd = mgr.Connection.CreateCommand())
        {
          cmd.CommandType = System.Data.CommandType.StoredProcedure;
          cmd.CommandText = "dsSelectList";

          int retryCount = 3;
          bool success = false;
          while (retryCount > 0 && !success)
          {
            try
            {
              using (var dr = new SafeDataReader(cmd.ExecuteReader()))
              {
                while (dr.Read())
                {
                  MyObject obj = MyObject .GetMyObject(dr);
                  this.Add(apt);
                }
                success = true;
              }
            }
            catch (SqlException ex)
            {
              if (ex.Number != 1205)
              { throw ex; }
              else
              { // 1205 Error = Deadlock Error
                //Custom Inhouse Logging

Log(LogTypes.Error, String.Format("A SQL Deadlock Occurred, I will retry this Query {0} more time(s).", retryCount-1), ex.Message);
              }
              this.ClearItems();
              Thread.Sleep(100);
              retryCount--;
            }
          }
        }//using
      }//using

      RaiseListChangedEvents = true;
      OnFetched();
    }

RedShiftZ replied on Friday, February 19, 2010

I misread your request. Ignore my answer. sorry.

cds replied on Friday, February 19, 2010

Thanks, your code is useful anyway.

RockfordLhotka replied on Monday, February 22, 2010

There's no good answer other than to modify CSLA itself so the TransactionScope wrapper class did your retry behavior. That's not ideal, but there's no other single location you can really plug into the pipeline to get this sort of result.

cds replied on Monday, February 22, 2010

Hi Rocky

Thanks for the answer.

I had a look at the code and it seems that DataPortalSelector would be the most convenient place to code this, if I really want to modify CSLA to allow this. I might give it a shot - it would be easy enough to catch the SQL Exception and check whether it was a deadlock and then retry, though to make it generic would be a bit more of a task.

Craig

 

RockfordLhotka replied on Monday, February 22, 2010

Yes, that's the spot.

One of my original goals for CSLA 4 was to open the pipeline more, including that location, to allow for more extensibility for these advanced scenarios. Unfortunately I'm running out of time, and that enhancement will probably not make it into the release Sad

cds replied on Wednesday, May 19, 2010

I finally got around to implementing this (the modification to CSLA to automatically retry after a deadlock). Here's my code, in case it helps somebody else.

 In the class Csla.Server.DataPortalSelector, I've added 2 new classes:

 

// NOTE: CDS modified CSLA
private static class ExceptionRetryDecider
{
  public static bool ShouldRetry(Exception ex)
  {
      var sqlEx = ex as SqlException;
      return (sqlEx != null && sqlEx.ErrorCode == 1205);
  }
}

private static class Retrier
{
  private const int MaxTryCount = 5;

  public static DataPortalResult DoWithRetry(Func<DataPortalResult> func)
  {
      var tryCount = 0;
      Exception lastEx = null;
      while (tryCount < MaxTryCount)
      {
          try
          {
              return func.Invoke();
          }
          catch (Exception ex)
          {
              lastEx = ex;
              if (ExceptionRetryDecider.ShouldRetry(ex))
              {
                  Debug.WriteLine(string.Format("Got exception for retry: {0}, try count = {1}", ex.Message, tryCount));
                  tryCount++;
              }
              else
                  throw;
          }
      }
     
      throw new Exception("Failed after trying " + tryCount + " times", lastEx);
  }
}

 

Then in each of the Create, Fetch, Update and Delete methods on DataPortalSelector, I've modified the code to use the Retrier class. For example, in the Fetch method:

public DataPortalResult Fetch(Type objectType, object criteria, DataPortalContext context)
{
  try
  {
      return Retrier.DoWithRetry(() =>
                                     {
                                         context.FactoryInfo =
                                             ObjectFactoryAttribute.GetObjectFactoryAttribute(objectType);
                                         if (context.FactoryInfo == null)
                                         {
                                             var dp = new SimpleDataPortal();
                                             return dp.Fetch(objectType, criteria, context);
                                         }
                                         else
                                         {
                                             var dp = new FactoryDataPortal();
                                             return dp.Fetch(objectType, criteria, context);
                                         }
                                     });
  }
  catch (DataPortalException)
  {
    throw;
  }
  catch (Exception ex)
  {
    throw new DataPortalException(
      "DataPortal.Fetch " + Resources.FailedOnServer,
      ex, new DataPortalResult());
  }
}

 

So, a fairly simple modification, but it would be nice not to have to modify the framework code Smile and hopefully Rocky can do something that's easily pluggable in a future version of the framework.

Craig

Copyright (c) Marimer LLC