CSLA .NET 2.0 Performance Issues

LDSK posted on Tuesday, December 30, 2008

I am using CSLA 2.0 with .NET 2.0.

On my win forms based application, I have a custom datagrid (designed by our team), which saves user preferences (sorting, filtering, columns width, hiding columns etc.,). To have sorting and filtering  on the datagrid on multiple columns, team decided to use ObjectListView by converting the datasource to a objectListView (as per my understanding this was found online via some forum on CSLA( may be this one..i dont know).

datagrid.datasource = New CSLA.ObjectListView(myListObject)

This datagrid loads around 15,000 - 20,000 rows at one time.

This has a huge performance hit. My Win form takes nearly 2-3 minutes to load.

When i debug i see that, though the CSLA objects takes about 30-40 seconds to fetch the data from the DB and assign to the datagrid, it then loops on public properties like for ever...I am unable to get past debugging this...

As you can see there are many variables to my problem. I am just wondering if the answer to this will be an upgrade to CSLA 3.0 as it promises performance increase, atleast to a certain extent.

I am also thinking about a third party datagrid control to further improve the performance.

It will be of help if some one can shed light on the performance improvement part of CSLA 3.0.

Will an upgrade to CSLA 3.0 improve performance signifiacntly?

Does anyone have the same issues?

 

dlambert replied on Tuesday, December 30, 2008

I can't tell you if CSLA 3.0 will handle this any better than CSLA 2.0, but I can see a number of areas where you may want to reconsider design choices.

First, 15,000+ rows of anything in a UI seems like a problem.  Nobody sits down at an application with the intent of reading 15,000 of *anything*.  What are your users really trying to accomplish here?  Are they scanning 15,000 items looking for a couple of items they're interested in?  Are they sorting / searching?  Are they processing a list from one end to the other?  If you know what your users are really doing here, you'll have a clue how to help them get to a more reasonable subset of items, and that'll be a big help.  From your description, I'd start with the filtering bit, and see if you could change the UI to find out what you're going to filter on first, and then narrow your query by that right up front.

If it turns out that you really need to display really large numbers of records, then keep reading.

The next bit that might bear investigation is the use of the phrase "one at a time" in your post.  If you're actually doing 15,000 DB calls, then there's a humongous improvement to be had by simply knocking that off.  Load a collection with one DB call, then pass the Data Reader down to children so that they can set their private properies.  I'm almost certain the Project Tracker app shows this technique.

On the subject of DB changes, you could also look at letting the DB help you with paging, so you're bringing back hunks of records (maybe 100 or 200 at a time).  This could help you in a couple ways.  If your grid supports paging, you can just let it ask for more records when it needs them, or you can load your collection asynchronously, and start displaying records as they arrive at the UI.  This is going to be more work than the method you're using now, but if you've really got one of those one in a million apps that needs thousands of records at the UI, you're going to have to work a little bit to make the UI responsive.

Finally, I'd look at the properties of the object itself.  Make it absolutely as narrow as possible.  This helps all the way up and down the stack by not hauling around information you're not going to use.  If you need to show another screen when someone double-clicks an object, fetch the "big" object then -- don't haul around 15,000 records worth of extra properties just in case the user clicks on #11,451.

Hopefully, there's something in here that'll give you some ideas.

rsbaker0 replied on Tuesday, December 30, 2008

I'm not sure this is a CSLA problem. I certainly agree that showing the user tens of thousands of rows is not a good way to do things, but we can easily fetch upwards of 10-20,000 rows/second from a SQL database into a  grid, even faster with a narrow table, and I'm not even using a native DAL but instead letting an ORM generate my SQL and fetch objects for me.  

You mentioned looping on public properties, which I could see being normal if this is just your grid trying to paint itself and fetching the values from your objects.

I'd consider trying a profiler to see where your code is spending it's time. (I test drove the ANTS profiler last year and liked it, but I'm sure there are other good ones).

 

lalsteris replied on Tuesday, December 30, 2008

In the case of csla 2.0, ensure that you are not loading your object values through your properties.  This will slow things down since the property setters usually check for authorizations and may fire change events if you haven't paused the firing of events.  You should aim for loading the values directly into the fields.  If you are doing this 20,000 times, you will experience significant slow down.

I have also experienced slow-loading data grids in the past ... if I can remember correctly, debugging pointed me to an ErrorProvider that was bound to the same bindingsource as my grid.  I'd have to look at some old code to remember exactly what I did ... if you temporarily remove any errorproviders from your form and it then loads a lot faster, then that may point to your problem.  Otherwise, I am totally wrong! 

Also, how does the "ObjectListView" work.  If it scans through and reads all properties of all records, then I assume the authorization checks are being called at least num_of_properties_per_record * number_of_record times.

Good luck.

Leigh

maxal replied on Wednesday, December 31, 2008

Creating 15000-20000 instances of CSLA objects will take time.

In one of our projects, where we used DevExpress grid, I implemented this solution:
DevExpress grid supports so-called "Server" mode. I implemented the "lazy" loading of objects to the list where first only primary keys were from the list were loaded. And then objects were loaded by request (with some buffers, of cause, but that's details). What I found is that loading only primary keys instead of creating objects was many-many times faster even when using same SQL statement. So, we achived very good results with that.

Other option we used for long lists are:
1. Creating and fetching objects in background process. Just make sure you move them to the main list in main thread. In fact, we used it in WPF application, but it doesn't matter, you can do the same in WinForms.

2. If you can, redesign you application to use pages, or Next <n> records approach. I know that many people will say that this is an obvious solution, expecially people from web world, but I also know that sometimes you can be stuck with arguments "my old program showed the list for me, why can't you do the same?" Good luck with that.

LDSK replied on Wednesday, December 31, 2008

Thank you all for your comments.

I am also investigating how to acheive paging on my Win data grid.

Paging is easy to do with a .NET object - Data Adapter has one implementation that lets you take the start records and max records. This implementation is not available on a CSLA.data.Objectadapter.

da.Fill(ds,startrecord,maxrecords,srctable)

maxal,

Can you please give me some sample code or example of how you did the lazy loading with Dev express grid ?

JoeFallon1 replied on Wednesday, December 31, 2008

I used the ObjectListView in 2.0.

In my view it is obsolete in 3.6. Because now we have LINQ to CSLA with indexing. I plan to fully remove it from my project next week and learn the new technology at the same time.

If I recall correctly the ObjectListView does have a performance hit for a large number of items. I think I saw it for 1,000 items. I am pretty sure it is a big part of the hit you are seeing with 15,000 items.

Joe

maxal replied on Friday, January 02, 2009

It was about two week of coding. I will probably take some time to describe it in details in by blog.

I even modified CSLA sources to support it. I could do it without such modifications but it would require a lot of duplicate code for read-only and not read-only lists. But briefly, I just supported IListServer interface from DevExpress.

LDSK replied on Friday, January 02, 2009

Maxal,

Thank you. please do let me know how to get to that post when you do so.

 

maxal replied on Sunday, January 11, 2009

Here it is, Lazy Loading for CSLA lists. http://discoveringdotnet.alexeyev.org/2009/01/lazy-loading-for-csla-lists-to-support.html

JonnyBee replied on Friday, January 02, 2009

Hi,

If your Property Get method include the following line
        CanReadProperty(true);
then your code (or actually CSLA)  makes a expensive reflection call to determine the property name.

For a start (and if you don't need property authorization) you cold just do a search and replace to add a comment tag before the CanReadProperty and CanWriteProperty methods. Then check the performance again and you will be surprised.

If you do need the property authorization you should include the property name to these methods. Then CSLA will not need to do the reflection call. Make your code like this and you will get a much better performance.

    public string Name
    {
      [System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
      get
      {
        CanReadProperty("Name", true);
        return _name;
      }
      [System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
      set
      {
        CanWriteProperty("Name", true);
        if (value == null) value = string.Empty;
        if (_name != value)
        {
          _name = value;
          PropertyHasChanged();
        }
      }
    }

And - If you have many ValidationRules being called for each of the items on Fetch from database they will also execute reflection calls and require some time. I

BTW - Csla 3.5 introduced the PropertyInfo and new Set/Get methods that avoids the use of Reflection.

/jonny

LDSK replied on Friday, January 02, 2009

Jonny,

Thank you for your reply. I searched my APP and found that CanReadPropert(True) is in every id property that uniquely identifies a record/object.

See below for property code.My APp does not implement any authorization for now. But i strongly beleive that who ever put it in there did so for a reason. Why do you think this line of code is in there?

<System.ComponentModel.Browsable(False)> _

Public ReadOnly Property Id() As Integer

<System.Runtime.CompilerServices.MethodImpl(Runtime.CompilerServices.MethodImplOptions.NoInlining)> _

Get

CanReadProperty(True)

Return id

End Get

End Property

Curelom replied on Friday, January 02, 2009

It could be the previous author used a template to generate the Properties and speed wasn't important at the time, i.e. they may have been working with less records.  I think it would be an extremely rare case that you would need to put read authorization on an id.  Check with the business needs of your object.  If you don't have documentation, or the previous author isn't available, then contact your business owners and document it for the next developer.  If you are going to be making any modifications at all to the code, you should be familiar with these. The walking on glass approach won't achieve anything.

LDSK replied on Friday, January 02, 2009

Curelom,

I appreciate your technical advice and not the rest.

Curelom replied on Friday, January 02, 2009

Sorry about that.  I'm often confusing to myself, let alone other people.  I'm just saying that the previous developer may not have had an actual reason for adding the authorization, such as using a template or because it's just a rather standard call.  You mention yourself that you aren't using authorization.  It seems that you are trying to read his/her mind.  I think all that will do is cause you to pull your hair out.

LDSK replied on Friday, January 02, 2009

dlambert replied on Friday, January 02, 2009

The CanReadProperty(True) syntax is pretty common -- it's the default syntax if you use the 2.0 code snippets, for instance.  It's slightly easier to code (especially if you're generating code) because you don't have to use the name of the property in the call.  It also withstands refactoring a little easier than the use of hard-coded property names because the C# refactoring tools won't pick up the string constant if you change the name of one of your properties -- you'll have to do it yourself.

All that having been said, I think Jonny's point is excellent - it's surely worth trying without that call to see how much of your problem is in that call.  It's very likely that you could do without the call in this case -- it's really only used when you've got a single object serving different types of users, and some users get to see more properties than others.  For example, if you've got an employee object and you want to hide the Salary property unless the user is in the "HR" group, you'd use this functionality.  Many objects don't need this kind of granularity.  If you're using this, you should be able to see evidence of it in your code -- look for "AuthorizationRules" with any of these property names, I believe.

If you can try it without the call, you'd know pretty quickly whether it's worth looking into further.  Even if you do need it, you should be able to pick up some time switching to the hard-coded syntax as Jonny pointed out.

LDSK replied on Friday, January 02, 2009

dlambert,

Thank you. I am going to try by commenting this line of code and will let you all know. Unfortunately this has to wait till Monday, as i am not in office today.

JonnyBee replied on Friday, January 02, 2009

Hi,

Yes, by default the property snippets for CSLA 2.0 includes those lines and fro the benefit of all those that want to use authorization. We also tend to override the CanWriteProperty methods where certain properties are only allowed to be set when some conditions are met (other then just authorization) such as a CheckBox, type or combination of other values.

We also had a case with slow performance this summer that was resolved with adding the property name or just removing the check for CanRead.. and CanWrite....

BTW: The CanRead.. and CanWrite.. lines are also the ones that requires the  compiler directive
 [System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
to make sure reflection will get the correct propertyname.

Did your company write an all new datagridview control or subclass the standard datagrid view. There are some issues with the standard datagridview and performance - especially with the AutoSizeColumn setting. You will find valuable information in the DataGridViewFAQ available at http://windowsclient.net/downloads/folders/applications/default.aspx

/jonny

JoeFallon1 replied on Monday, January 05, 2009

JonnyBee:

Hi,


BTW: The CanRead.. and CanWrite.. lines are also the ones that requires the  compiler directive
 [System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
to make sure reflection will get the correct propertyname.

Note:

The use of reflection to get the property name was found to be unreliable and was made obsolete. The string value of the property name should be provided in old code.

Joe

 

ajj3085 replied on Monday, January 05, 2009

Also, if the CanReadProperty is going to be removed, or if it's going to be modified to include the string literal property name, I would remove the System.Runtime.CompilerServices.MethodImpl attribute.  Inlining would allow the method call to be sped up, IIRC.

RockfordLhotka replied on Monday, January 12, 2009

Even upgrading from 2.0 to 2.1 will probably help a lot. In fact the whole point of 2.1 was performance improvements, due to some poor design choices on my part in the 2.0 implementation.

These specifically impacted business/validation rules and authorization rules. The switch from per-instance to per-type rules had a radical impact on both performance and memory consuption.

But if you are going to upgrade, I'd recommend going to 3.0, because it includes more bug fixes and so forth beyond 2.1, along with the performance enhancements.

LDSK replied on Tuesday, January 13, 2009

Hi Rocky,

Thank you. One more question. If i decide to upgrade 2.0 to 3.0, do i have to make any code changes to my application at all?

Do i have to make any code change once the upgrade is done to make it work with the new version?

thanks,

Suhanya

RockfordLhotka replied on Tuesday, January 13, 2009