DataPortal CPU spike

DataPortal CPU spike

Old forum URL: forums.lhotka.net/forums/t/9708.aspx


ChaseC posted on Thursday, November 04, 2010

Hello all,

We have a winForms smart client application using CSLA 2.1.4 that has been running for a few years now.  Within the past week or so, our dataportal process has starting using 100% CPU two or three times a day. 

We've had a lot of cooks in the kitchen lately, so debugging has been interesting to say the least.

Any ideas on how to troubleshoot this?  I found this thread that mentions large object graphs being de/serialized: 

http://forums.lhotka.net/forums/p/486/2409.aspx#2409

Any other ideas?

 

 

 

 

 

rfcdejong replied on Friday, November 05, 2010

Hard to say without more information, it's an older csla version. Maybe you should think about upgrading the application using a newer version of csla.

ChaseC replied on Friday, November 05, 2010

Yeah, that is the plan.  Unfortunately we have to stabilize the legacy app before we can focus on the rewrite. 

What's happening is that the dataportal process is running fine for 4 or 5 hours, and then the cpu climbs to 100% within a minute.  It sounds like an infinite loop, but we haven't been able to isolate it yet. 

Our infrastructure guy started recycling the app pool every hour, which seems to minimize the impact for now.  But we still need to figure out the root cause.

I realize it's a vague scenario.  Is there any additional information I could provide that would help?

bniemyjski replied on Friday, November 05, 2010

Hello,

I would run this under a Performance Profiler like the one RedGate has. Also it would not hurt to add some Windows Performance Counters and check the Windows event logs for more information.

Thanks
-Blake Niemyjski

ajj3085 replied on Saturday, November 06, 2010

It sounds like you have a good workaround in place.  Perhaps just focus on doing the minimal work needed to move to a more recent Csla, and then test if the problem still occurs in when using the new version of Csla.  maybe even just get to 3.0.5 to see if that resolves it.

RockfordLhotka replied on Sunday, November 07, 2010

Odds are that it isn't CSLA code. It sounds like this was working for a long time, and recently started having the issue. So the most likely cause is some new business object code that's doing something bad - infinite loop, accidental recursion, etc.

If you have a custom base classes, you could insert logging code in the pre- and post-processing methods for the data portal, thus instantly gaining visibility into every data portal call.

Or you could modify Csla.Server.DataPortal to do pre- and post-processing logging. (something that's on the wishlist for 4.2 perhaps)

ChaseC replied on Sunday, November 07, 2010

Oh yeah.   Pretty certain CSLA isn't the cause.  I  didn't mean to imply that.  It's just been difficult to pinpoint the problem because so much is going on within the remote dataportal and we haven't been able to reproduce the problem locally.  

For now, we're going to keep recycling the app pool so that we can get to work on the new version.  Although it seems like you'd be just as likely to hit an infinite loop within an hour?  I don't really understand how that keeps us running.   Whatever works I guess.  

Many thanks to everyone for the suggestions.  

Copyright (c) Marimer LLC