CSLA performance vs "Old school" plain object implementation

CSLA performance vs "Old school" plain object implementation

Old forum URL: forums.lhotka.net/forums/t/7341.aspx

Marius posted on Sunday, July 26, 2009

I have been playing around with the CSLA framerwork and I must admit that it has some amazing features. However, I tried to understand the performance drawbacks of implementing this framework and below are the results of  a small test that I did:

Performance test details:
 - Database: SQL server 2005
 - CSLA framework: version 3.7.0-090721
 - test machine: Windows Vista x86 / 3GB RAM/ IIS 7
I implemented a small business object (10 public fields) with no children objects or collections of children objects and no validation or authorization rules defined. Also, "LINQ to CSLA" and the WcfPortal was used for saving/manipulating the data (one DB table with primary key defined on the ID field). The fields were a mixture of managed backing fields and private backing fields.
In the same time I created a plain C# object with the same number of fields and a method to save the data using the same stored procedure used also for the CSLA object implementation.
Note: the test client, CSLA framework and the DB are on the same developer machine.

The results are below:
CSLA Business Object with WcfPortal implementation (taken from PTracker)
0 records written in 00:00:07.0850000 (this is the time to initialize the WCF service and the relevant proxy/channel and security checks)
100 records written in 00:00:13.7280000
200 records written in 00:00:20.6710000
300 records written in 00:00:27.1620000
400 records written in 00:00:34.0160000
500 records written in 00:00:40.6420000
600 records written in 00:00:47.7020000
700 records written in 00:00:54.5670000
800 records written in 00:01:01.1580000
900 records written in 00:01:07.6420000
1000 records written in 00:01:14.2790000

POCO (Plain Old C# Object) with direct Save method to the DB via a stored procedure
0 records written in 00:00:00.2070000 (time to create the DB connection to the DB)
100 records written in 00:00:00.3650000
200 records written in 00:00:00.4470000
300 records written in 00:00:00.5280000
400 records written in 00:00:00.6070000
500 records written in 00:00:00.6850000
600 records written in 00:00:00.8060000
700 records written in 00:00:00.8930000
800 records written in 00:00:00.9700000
900 records written in 00:00:01.1520000
1000 records written in 00:00:01.2460000

Result: 1000 small business objects are written in 74.2 seconds using CSLA and in 1.2 seconds using plain objects (every 100 objects are written in about 6-7 seconds using CSLA and about 0.10-0.15 seconds using POCO implementation)

The same test was performed against a remote SQL Server 2005 (everything else is on a different machine)
CSLA Business Object with WcfPortal implementation (taken from PTracker)
0 records written in 00:00:07.0540000
100 records written in 00:00:43.0070000
200 records written in 00:01:28.6990000
300 records written in 00:02:18.2750000
400 records written in 00:02:56.7490000
500 records written in 00:03:31.7450000
600 records written in 00:04:07.4880000
700 records written in 00:04:49.3700000
800 records written in 00:05:29.7510000
900 records written in 00:06:14.8260000
1000 records written in 00:06:58.0680000
POCO (Plain Old C# Object) with direct Save method to the DB via a stored procedure
0 records written in 00:00:00.7030000
100 records written in 00:00:10.8050000
200 records written in 00:00:23.5430000
300 records written in 00:00:33.6890000
400 records written in 00:00:45.1590000
500 records written in 00:00:55.2050000
600 records written in 00:01:06.9380000
700 records written in 00:01:22.3810000
800 records written in 00:01:46.2080000
900 records written in 00:01:59.4900000
1000 records written in 00:02:09.7710000
In this case every 100 records are written in the remote DB in about 40-45 seconds using the CSLA framework implementation and in about 10-12 seconds using POCO implementation.

I need to mention that the implementation of the business objects using CSLA was done following the PTracker example.

The question that I have for the people with more experience in the CSLA implementation is:  putting aside the benefits of this framework is this performance expected or the implementation that I did may not be aligned with the CSLA implementation guidelines?

Thank you!

Marius

RockfordLhotka replied on Monday, July 27, 2009

You are comparing radically different scenarios here - 3-tier vs 2-tier. You shouldn't be surprised that adding an extra network hop and all the attendant overhead causes a substantial performance impact.

When I give talks about distributed computing I often say "the more tiers you have, the more tears you'll have". This is one clear reason - the cost of adding each tier boundary is quite high, and should only be done because you have a clear business need.

If you'd be happy with the 2-tier model you are using with your POCO scenario, you might consider using a 2-tier scenario with CSLA .NET to see if it meets your needs.

I am sure it will still be slower, because CSLA is doing a lot of useful things you don't get with a POCO model. You should evaluate whether the POCO model gets you what you need, or whether you need full support for data binding, a level of in-memory transactional modeling for your objects, structured business, validation and authorization rules, etc. If you don't need those things, then CSLA would provide little value.

If you do need those things, then you can't use POCOs anyway - you'll either need a framework, or you'll essentially end up recreating some of what CSLA does.

ajj3085 replied on Monday, July 27, 2009

In addition to what Rocky said, I'd also warn about getting stuck in a performance optimization anti-pattern. In other words, you're worrying about performance when it might not really be an issue.

In most applications of Csla, its highly doubtful your user would be editing even 100 objects within a single use case. 1000 becomes even more absurd. I can't imagine a good interface for allowing a user to easily edit 1000 which would then be submitted as a batch. More likely, the user will be concerned about a smaller subset of information to work with.

Certainly there may be cases which large amounts of data are edited in one shot.. but I suspect these would be edge cases, not your normal use case. I wouldn't get too hung up on performance metrics.

Just my 2 cents.
Andy

Marius replied on Monday, July 27, 2009

As I mentioned in the initial post I don't question the rich set of features that CSLA framework has or implements. Also I am very aware that I'm not comparing apples to apples here - it's more like an orange to a sesame seed comparison. In the real world systems you hardly find one which is using only POCO objects. My expectations (as everybody else's) are  that CSLA performance (or any rich framework) is lower than a POCO performance due to the overhead of having a set of rich features.
I am more interested in finding out what is the response time of a operation like "Save" when an average size business object is new (therefore triggering an insert in the DB) in different production settings like 2/3/4 logical or physical tiers. Does anybody have any benchmarks that can be shared here? I know I'm asking too much as there may be zillions of other variables, but it is a starting point in creating projections/estimations for system performance.

My test would probably not be a real business scenario where one user is creating 100 new business objects, but a real world scenario is when 100 concurrent users are creating 1 business object - it's not exactly the same but it puts a similar load on the system (if it is not a clustered environment of course).

The reason for being interested in this is the following: in many systems there are set indicators like Quality of service which may be calculated as the amount of time an user is spending to complete an workflow (let's say creating an order in the system from start to checkout). The user experience is divided in two:
   1. Think time - the time that the user needs to figure out what actually is needed to be pressed/entered in the GUI to complete the workflow - in this discussion thread I am not interested in this aspect.
   2. System time which is calculated as the total time that the user needs to wait for the system to do the background processing: creating new objects/updating existing objects/ saving the data/ calling external interfaces. When you add the time to perform these operation you get to a figure that makes the difference between night and day.  There are systems where each second saved in the system time translates in cost saving of  around 100.000$. Needless to say, during an average workflow I am expecting to have somewhere between 20 and 50 business objects created and saved and therefore it will be a difference if this operation is completed in 1-2 seconds or 20-30 seconds.

Thank you!

Marius

Wbmstrmjb replied on Monday, July 27, 2009

I don't have hard numbers for you, but we have some very large BOs with multiple child lists, etc and the performance is a non-issue even with remoting from the web server to an app server. Like mentioned earlier, most "Saves" are single object or maybe a couple of objects and to do "batches" would mean that you should have an SP or some other means to make mass updates, not a for loop with a couple hundred BOs. If you're looking at CSLA as a possibility, I wouldn't question performance. Like Rocky said, you either have a need for what the framework (any framework) offers, or you don't. CSLA isn't ever going to be used to brute force crack NSA's toughest encryption and so insignificant "benchmarks" are not useful.

Wbmstrmjb replied on Monday, July 27, 2009

Marius:
My test would probably not be a real business scenario where one user is creating 100 new business objects, but a real world scenario is when 100 concurrent users are creating 1 business object - it's not exactly the same but it puts a similar load on the system (if it is not a clustered environment of course).


Also, this is not even close. In this case, your DB and network have far more to do with performance than CSLA or any other framework.

andy replied on Monday, July 27, 2009

I just want to add the maintenance cost factor to it.  It's far out-weigh the extra overhead.

Andy

Marius replied on Monday, July 27, 2009

Thanks guys - you've been very helpful. I now have a better picture of what to expect from CSLA performance-wise.

Marius

rsbaker0 replied on Tuesday, July 28, 2009

Here's another scenario for you to try that might provide for a more realistic comparison.

With CSLA, you really have to be mindful of both sides of the data portal and what should be done client side versus server side. Anything that involves multiple round-trips to the database is a good candidate for "server-side" processing. Note that with 2-tier deployment these are both happening on the same client, but you still have to plan for 3-tier.

You can also optimize your server-side processing to avoid the data portal overhead. If a method can only execute server-side, there is no need to make a data portal call (it works and basically works like a 2-tier scenario, but it adds overhead).

So, in the intermediate classes that all our CSLA objects derive from, a direct Update(TransactionContext) method that takes the actual transaction in progress is available for internal use server-side only. This is much closer to a native update than making a Save() call from the client. We are actually quite pleased with how fast this is.

Try sending a BusinessListBase class of 100 objects over to the server, and have the server-side processing save them without making data portal calls (child data portal calls are probably OK but i haven't used them).

Any complex object that does multiple updates would similarly be written to do these server side.

RockfordLhotka replied on Monday, July 27, 2009

If you discount the time required to instantiate and populate new objects with data (which is all generally absorbed by the “think time” category), then you really must simulate the load pretty closely to get a decent projection.

 

2-tier vs 3-tier becomes a really big deal, because the serialization, network transfer and deserialization costs for 3-tier add up fast. 2-tier doesn’t have that to the same degree, though the data portal does (optionally) serialize/deserialize the object graph to ensure the object graph remains valid in the case of a database exception.

 

(In very high transaction scenarios (where exceptions are unlikely) you’d almost certainly turn that off, because it would be cheaper to rehydrate the object graph from the database (losing all user changes of course) than to clone the graph.)

 

My point is that you were comparing 2-tier to 3-tier. That’s a totally invalid comparison, regardless of rich vs plain objects.

 

Your tests would be reasonable if you either switched the data portal to 2-tier mode, or put a web service between your client and your database in the POCO scenario.

 

But also, you suggest that doing one save of 100 objects can simulate 100 users saving 1 object. In a 2-tier model you might get away with that, but in a 3-tier model that would be very inaccurate, because you would only count 1% (roughly) of the client-to-server communication costs. The only way to get any reasonably valid data in an 3-tier model (or 4- or higher tiers) is to actually test in that model – which gets somewhat complex because you have to effectively simulate concurrent users, and not saturate the network stack on the client (where you can get unexpected blocking issues due to all the nice anti-virus and anti-worm features in Windows).

 

Fwiw, when I do perf testing for CSLA itself, I don’t use a database. I use in-memory mock data. The problem with using a database is that there are 6-8 technologies to talk to SQL Server, and many variations on how to use those technologies. Any test that talks to an actual database is only valid within the parameters of the technology and technique chosen.

 

Besides, actual database perf testing with different technologies/techniques can be easily accomplished in isolation. And probably should be – so you can decide whether to use raw ADO.NET, L2S, EF, Astoria, TableAdapter, etc. on their own merits.

 

Rocky

ajj3085 replied on Tuesday, July 28, 2009

I don't have any hard numbers, but i have a screen where the user edits an invoice. The invoice is the root BO, there's always three other child BOs, and then one BO per line item. A large order (70+ line items), using .Net Remoting with compression in a 3-tier setup (client to app server, and the database server is on the same physical machine as the app server) takes less than 2 seconds from the time the user clicks save until the user can again use the screen. This is also across a T1 link between buildings (some users are in another town about 20 minutes away), and includes enabling / disabling all the controls on the form (there's 4 tabs, about 80 controls, some of which are 3rd party, on the form).

Other, simplier use cases the user can save and its pretty much as quick as saving a local Word document.

jh72i replied on Friday, November 06, 2009

ajj3085:
..using .Net Remoting with compression...

Hi, just reading this post and this caught my eye. Don't want to muddy this thread but could you please let me know what you use to achieve this compression? And whether it can be applied using configuration if an app for one set of users used remoting but for another set doesn't.

Thank you.

ajj3085 replied on Tuesday, November 10, 2009

I think if you search the forum you should be able to find the same code I used.  Its called CompressedDataPortal or something like that. 

RockfordLhotka replied on Tuesday, November 10, 2009

http://www.lhotka.net/cslanet/faq/DataPortalFaq.ashx

relawson replied on Wednesday, November 04, 2009

Although the test in this thread isn't an apples to apples test, I am still interested in how POCO vs CSLA compares.  One could argue that CSLA is not tightly coupled to any ORM so pick the best ORM and "plop" CSLA on top.

I don't think the best measure of CSLA is with persistance since it really isn't about how you get or update data, rather how you present data to the developer in the form of an object.  The best measures are, IMHO, how quickly developers can implement CSLA, how well it scales, and how extensible it is.

The CSLA examples I have seen cause me some anxiety because the business layer is tightly coupled with the DAL - and everything is in the same class.  And I've yet to seen an (I)nterface.  Perhaps I have seen poor examples.  Are there any examples of what could be considered "elegant" implementations of CSLA?  It was an effort to create the relationships between parent and child objects (lots of keystrokes) - something I would expect the CodeGen template to do for me based on the database schema.  Perhaps there are some better CodeSmith or T4 templates out there?

Although I see the convenience of the mobile object, I think it also introduces complexity.  The binding examples I have seen are quite complicated, at least in winforms.  That complexity goes away in WPF and Silverlight - but that is because those technologies make binding much easier.

Anyone have a sample project that is the "posterchild" for CSLA and is also "real world" type of application?  Any really good CSLA templates out there?  At the end of the day what really matters is how quickly and how well I can deliver a solution to a business problem.  I doubt the Holy Grail exists (RIA Data Services?) but before I write off CSLA I would like to see the best examples of how it can be implemented.

ajj3085 replied on Thursday, November 05, 2009

relawson:
The CSLA examples I have seen cause me some anxiety because the business layer is tightly coupled with the DAL - and everything is in the same class.  And I've yet to seen an (I)nterface.  Perhaps I have seen poor examples.  Are there any examples of what could be considered "elegant" implementations of CSLA?  It was an effort to create the relationships between parent and child objects (lots of keystrokes) - something I would expect the CodeGen template to do for me based on the database schema.  Perhaps there are some better CodeSmith or T4 templates out there?

As far as tightly coupled DAL, check out the ObjectFactory.  Its there to enable a looser coupling to a DAL.  There is also plenty of interfaces which exist in Csla for you to use, and you can of course make your own. 

I'm not sure what you're getting at with the keystrokes measurement; if you're using it to measure developer productivity, you're going to be disappointed in the same was as if you were to use KLOC.  Its just not an effective measure of productivity.

relawson:
Although I see the convenience of the mobile object, I think it also introduces complexity.  The binding examples I have seen are quite complicated, at least in winforms.  That complexity goes away in WPF and Silverlight - but that is because those technologies make binding much easier.

Yes, data binding is easier in Wpf and Silverlight, but Csla does offer some classes to help with WinForms, such as BindingSourceNode, CslaActionExtender, etc.

relawson:
Anyone have a sample project that is the "posterchild" for CSLA and is also "real world" type of application?  Any really good CSLA templates out there?  At the end of the day what really matters is how quickly and how well I can deliver a solution to a business problem.  I doubt the Holy Grail exists (RIA Data Services?) but before I write off CSLA I would like to see the best examples of how it can be implemented.

I don't have a sample nor can I post code for what I have, but I have an application based on Csla which is in real production use.  Its a simple CMS, quoting, ordering (not fulfillment), invoice, product, and software serial number system, which ends up exporting to accounting software.  Basically, the application covers the whole sales workflow from quote to sending invoices, and includes about two dozen reports... all built on top of Csla.

RockfordLhotka replied on Thursday, November 05, 2009

For some areas where CSLA .NET is used you can look here:

 

http://www.lhotka.net/cslanet/usage.aspx

 

Wbmstrmjb replied on Thursday, November 05, 2009

RockfordLhotka:
For some areas where CSLA .NET is used you can look here:http://www.lhotka.net/cslanet/usage.aspx


You can add "A Department of the State of California" to your list Rocky.

RockfordLhotka replied on Thursday, November 05, 2009

Thank you!

-----Original Message-----
From: Wbmstrmjb [mailto:cslanet@lhotka.net]
Sent: Thursday, November 05, 2009 4:49 PM
To: rocky@lhotka.net
Subject: Re: [CSLA .NET] RE: CSLA performance vs "Old school" plain object
implementation

RockfordLhotka:For some areas where CSLA .NET is used you can look
here:http://www.lhotka.net/cslanet/usage.aspx

You can add "A Department of the State of California" to your list Rocky.

Marjon1 replied on Wednesday, November 11, 2009

While we mentioning companies & products, please feel free to add Sybiz Software (http://www.sybiz.com) to the list of companies using your fantastic framework!.

We are using it to build accounting, payroll & contact management products.

rsbaker0 replied on Thursday, November 05, 2009

relawson:

Although the test in this thread isn't an apples to apples test, I am still interested in how POCO vs CSLA compares.  One could argue that CSLA is not tightly coupled to any ORM so pick the best ORM and "plop" CSLA on top.

...


Incidentally, this is exactly what we did to implement our application. We use CSLA to provide for object mobility, data validation, and bindability and let the ORM handle the details of fetching/persisting the data.

Were I to start over again, I'd still be willing to use an ORM, but would look at it more with the idea of having multiple classes associated with the same database table(s). It's easy to misuse CSLA and build objects that are too fat because you have combined multiple use cases into the same class.

All in all, I think we're paying about a 30% penalty in fetch performance versus using a native dataset (this is mostly in the ORM), but we have offset this by implementing paging support in all of our root object grids. So, we get 2 second or less response in most cases displaying a grid even if the underlying database table has 100,000 rows or more.

The legacy app was C++, POCO with ODBC for reading/writing, and we have no complaints with CSLA performance thus far, as long as you keep your objects caparatively lean.

Copyright (c) Marimer LLC