Binding a Progress readout to a large collection fill

Binding a Progress readout to a large collection fill

Old forum URL: forums.lhotka.net/forums/t/5948.aspx


bcrowell posted on Saturday, December 06, 2008

Can anyone help me with how to accomplish this concept?

I've got a large csla collection object I need to fill on an app server. Speed is not such an issue as this is really a disaster recovery sort of operation. However, I would like to have a progress readout showing the percent completed of the collection load as it fills up. I tried disabling the Me.RaiseListChangedEvents statement the refresh based on the bindsource's listchanged event but that didn't get me anywhere. Any help would be appreciate. Thanks all.

Byron

bcrowell replied on Saturday, December 06, 2008

FYI this is a winforms app using csla 3.5. Thanks.

RockfordLhotka replied on Saturday, December 06, 2008

Unfortunately this is relatively difficult. Even if you look at it in a 2-tier model it is really hard, and 3-tier makes it worse. Well, maybe not, since the basic answer is the same in both cases...

Problem 1 is that ADO.NET doesn't raise events as it loads the data, so unless you break the database query into chunks (so you do multiple queries to get multiple subsets of the data) you'll have to wait until all the data is retrieved from the database.

Problem 2 is that you then copy the data retrieved from the database (now in a datareader, dataset, entity collection or whatever) into your business object. This can be done using a looping structure, and you may be able to run a progress bar off this - but for two things. First, if you use a datareader you don't know how many rows you are going to get until you reach the end. Second, in a 3-tier model this code is running on the app server.

Problem 3 is that you want the progress bar to update while the processing is occurring, which probably means running the data load on a background thread. Though you can use tricks with Timer controls to avoid this.

The solution is to do the multiple queries - chunking your data retrieval - but this must originate from the client workstation so it can do the progress updates. Before going further you should realize that this will not perform as well as getting the data in one shot, but it will allow for meaningful progress display.

(as an aside, this is the reason most Microsoft software doesn't show real progress - it is too expensive in terms of both performance and complexity - so they show a busy animation instead)

Anyway, here's the deal.

In your collection's static factory method, DO NOT directly call the data portal. Instead, you are going to be a little tricky.

  1. In your collection, implement IReportTotalRowCount - your UI can use this interface to get the total row count
  2. Create a private Command (RowCountRetriever) object inside the collection
  3. Write the DataPortal_Execute() method to retrieve the total number of rows of data
  4. Create a private Command (DataChunkRetriever) object inside the collection
  5. Write the DataPortal_Execute() method to retrieve a chunk of data
  6. In the collection's factory method, use RowCountRetriever to get the total number of rows to expect
  7. In the collection's factory method, in a loop, make repeated calls to DataChunkRetriever to retrieve all the data in chunks - each call goes to the app server, which talks to the database to retrieve a small chunk of the overall data
    1. When you get a chunk, disable event raising in your collection
    2. Add the new data into your collection
    3. Reenable event raising
    4. Manually call OnListChanged() with a Reset change
      1. The ListChanged event will cause the UI to update
      2. Your UI should directly handle ListChanged so it can update the progress bar
      3. Remember that the UI can use IReportTotalRowCount to get the max value for the progress bar

As I said, this doesn't perform as well, and it is obviously more complex than the direct approach. But it will get you a fully functioning progress bar in a 3-tier environment, and without multi-threading.

 

bcrowell replied on Saturday, December 06, 2008

Terrific as usual.
Thanks Rocky. I'll take a whack at this. It's pretty cool to get full support on a Saturday morning.

thanks.

Byron

sergeyb replied on Saturday, December 06, 2008

This is not as simple as it sounds.  Reason being the data is filled on the server, but you want to show the progress on the client.  The only way I could think of is to have the filling process update a row in progress table as it runs, using some sort of unique operation identifier as a key.  This key would be generated on the client prior to calling fill operation.  You would need to call fill operation on a background thread, not main thread of the application. On the main thread you would probably have a time that would fire a command object to get the value for progress table/row.  Once you get the value, you can update a progress bar in the UI.  Once you background thread completes, you can stop the timer and hide the progress bar.  You might also never complete the progress bar (set to 100 %) until background thread completes, as it may not complete even though the progress value in DB is 100 % due to transfer speed of the object. 

Does this make sense?

 

Sergey Barskiy

Principal Consultant

office: 678.405.0687 | mobile: 404.388.1899

cid:_2_0648EA840648E85C001BBCB886257279
Microsoft Worldwide Partner of the Year | Custom Development Solutions, Technical Innovation

 

From: bcrowell [mailto:cslanet@lhotka.net]
Sent: Saturday, December 06, 2008 9:45 AM
To: Sergey Barskiy
Subject: [CSLA .NET] Binding a Progress readout to a large collection fill

 

Can anyone help me with how to accomplish this concept?

I've got a large csla collection object I need to fill on an app server. Speed is not such an issue as this is really a disaster recovery sort of operation. However, I would like to have a progress readout showing the percent completed of the collection load as it fills up. I tried disabling the Me.RaiseListChangedEvents statement the refresh based on the bindsource's listchanged event but that didn't get me anywhere. Any help would be appreciate. Thanks all.

Byron


Copyright (c) Marimer LLC