Batch processing in CSLA 1.5 - requeing failed jobs?

lewischris posted on Tuesday, August 15, 2006

Hi,

We are using a rudimentary in-house MSMQ batch processing engine currently. I am keen to look at the batch processing engine from the .NET 1.1 CSLA framework but looking through the code there doesn't appear to be a mechanism to move or automatically requeue 'failed' request messages onto a 'failed jobs' queue?

Our engine currently requeues request messages on a 'pending investigation' queue if they fail for any reason. This allows us to find out what the failure was (possibly a data integrity issue, or possibly just a 'database was busy' timeout, etc.), if possible fix it and then re-submit the request.

Anyone have any comments or suggestions about this.

Thanks for any help.

Chris

rhoeting replied on Tuesday, August 15, 2006

The batch queue mechanism in CSLA 1.5 is pretty basic, but light years ahead of what you get out of the box on windows.  We use it quite a bit and see it as a good starting point under which new stuff can be easily added if need be.

Rob

lewischris replied on Tuesday, August 15, 2006

Thanks Rob.

How do you cope with batch requests that fail - do you just report the failure and move on (which I think is what the batch processor does) or are you able to recover failed jobs for further re-processing?

This is the key question for us, as we cannot afford to 'lose' failed batch request messages. As I mentioned, we currently have a batch processor that 'saves' failed jobs and allows them to be subsequently re-queued if necessary.

Chris

ajj3085 replied on Tuesday, August 15, 2006

Chris,

You should be able to add functionality to the batch processor to do this.  I think it makes sense to drop the message into an Investigation queue if the batch fails; just add this functionality in the catch portion of a try which wraps the execution of the job.

Andy

rhoeting replied on Tuesday, August 15, 2006

It's an ok solution, but not great.  We've created a "BatchBase" class which encapsulates the plumbing which simplifies the creation and execution of batch jobs.  One piece of this base class is a logging mechamism which persists info to a few simple tables.  So as jobs are started, completed, failed, information is logged.  The tables can be queried to determine success or failure.  If failure is detected jobs need to be manually submitted. 

Not ideal, but at the same time, it wouldn't be a huge leap to automate the checking and automatic resubmission of jobs.

Hope this helps.

Rob