20 seconds to load customer list of 4000 - help !

20 seconds to load customer list of 4000 - help !

Old forum URL: forums.lhotka.net/forums/t/4331.aspx


johnT posted on Monday, February 11, 2008

I am a beginner at vb.net and csla for that matter, so I need some very basic idea of how long a customer list of 4000 customers 'should take' to load from a SQL Server database via CSLA list object. Ours is currently taking over '20 seconds', and that's about '10 times longer' then the same amount of data takes to load in the old 'MS Access' application were trying to replace.

The sql server database is on the same workstation as the program etc., so its not our network either. However, we are working with CSLA Ver. 1.1, but I don't think that should cause any speed problems either, should it ?

Any ear marks for basic data loading times you can supply would be greatly appreciated. I'm pretty sure what I have now is way off normal, so any ideas on how to speed things up would be also be great.

thanks

webjedi replied on Monday, February 11, 2008

Check this post out to see if it helps: http://forums.lhotka.net/forums/thread/21038.aspx The long and short of this thread was that the objects were being loaded one at a time instead of through one statement...Not sure there is a connection without seeing any code.

However there were huge performance gains in version 2.1 and higher.  So you might want to look into going to at least 2.1...(if you can)

rsbaker0 replied on Tuesday, February 12, 2008

webjedi:
...The long and short of this thread was that the objects were being loaded one at a time instead of through one statement...

Yes -- when loading lists of objects the implementaton needs some internal methods (not CSLA-specific) to let it populate objects from a IDataReader-type interface. (This should all be happening server-side anyway).  CSLA  can serialize the entire resulting object list once it has been generated, but doesn't need to be involved in loading each individual object. If you're doing a round-trip per object, performance will be abysmal.

Also, if you can load the fields directly rather than using the Property setters when initially populating the objects, this will also help performance.

johnT replied on Wednesday, February 13, 2008

Thanks a lot people. The solutions you suggested seemed to have worked great. Our code was loading customer objects one at a time. Thanks to this forums help our applications loading speed for 4800 customers went from a 'v ..  e ..  r ..  y ..    s .. l .. o .. w.. '  35 seconds,  'tojust2'  !

This forum rocks .....  !     ;-) 

Q Johnson replied on Monday, February 11, 2008

I am still maintaining CSLA ver 1.1 apps and don't usually see the kinds of performance you're describing here.

General database issues

The one record at a time possibility mentioned in the other post here should be investigated and ruled out, to be sure.  After that, you'll need to look at the SQL or T-SQL techniques you're using to get that resultset of Customers.  4,000 records is not a challenge for SQL Server (any more than it is for Access as you noted), but there are lots of different ways to get them.  Do the queries involve a lot of complex joins and filters?  Worse still, might you being manipulating Cursors to process them?  These are both big performance killers.  But if you are doing a fairly simple Select statement against a Customer table (even with a single inner join to, say, an Addresses table), 4000 records should come your way in about half a second more than 20 do.

Are you using ADO.NET or are you just borrowing your ADO code and using the COM-based data access objects?  ADO.NET will beat it handily (but even ADO in .NET 1.1 should be doing better than you have described).

How are you measuring the time?

How are you deciding when the "load" is done?  Are you populating a control to display them all?  Might there be a problem with the way you are using that control?  I can get thousands of records and display them in a DevExpress grid in well under five seconds (and it scrolls them VERY quickly).  I really don't know what to expect from the built in datagrid and list controls, though.  Are you trying to use a complicated UserControl?  Perhaps the code in that control could use some attention.

Are you using Data Binding?

You mentioned that you're trying to upgrade from an Access-based app.  And if that was in VB6 (or earlier! <cringe>), the coding technique probably avoided the then-abysmal data binding techniques.  Are you populating an array one record at a time and determining the load time by the time it takes to finish popluating it (with the presumed goal of using Unbound data handling techniques in the UI)?  It was popular then to create TWO copies of an array - one for use for editing and the other to provide quick Cancel performance.  If there was any attempt at tier separation, you may have had the user interface send a Business Object a 'Get' command, which in turn may have sent the request to a Data Access layer that retrieved a recordset from the data store, populated the array and then passed it back to its caller (the Busn Obj), which would ultimately send it to the UI. 

Databinding in .NET is awesome.  You can eliminate most of the effort here by simply having your Customers collection's (EditableRootCollection in CSLA 1.1 speak)  DataPortal_Fetch routine populate itself and the User Interface merely makes an assignment of that collection of Customers to the .DataSource property of the target UI element (grid, listbox, or whatever).  It's VERY fast, flexible, and safe.

I hope one of these suggestions has you looking into the right section of your code.

Regards,

johnT replied on Tuesday, February 12, 2008

Thanks a lot for all the great info. It will take me a while to try and check it all out, but I will let the forum know how things turn out once we've tried the suggestions.

thanks again

Copyright (c) Marimer LLC