memory leak - part 2

memory leak - part 2

Old forum URL: forums.lhotka.net/forums/t/1317.aspx

mheere posted on Monday, September 25, 2006

Further to my earlier post of a memory leak and 'blaming' the CSLA framework I know have found out that this may be a pure Windows problem.
The problem is - and let me focus on the CSLA part - that when you repeatedly call for a Business Object (BO) that collects its data from various sources and passes it back to the client the memory you use increase. However, when this BO goes out of scope (and you do the Dispose() etc.) you would expect the memory to have gone down to what it was before.
However, what happens here is that I am having to repeatedly call a BO which therefore causes my app to run eventually out of memory. 
A remarkable thing to note is that if you minimise and then restore the app it will bring the memory right down again - and so it continues.

The amazing thing is that this seems to apply to all Windows apps.  Start Word, type away, check the Task Manager's memory consumption for this app, now minimise/restore the app and the memory will have come right down.

This behaviour seems to be known (http://www.dotnet247.com/247reference/msgs/35/178959.aspx) but what I don't understand is that this phenomenon is killing my app so this MUST also happen to other people!

Has anyone got any suggestions?

Cheers

Marcel.

 

 

DansDreams replied on Monday, September 25, 2006

Need some clarification here.

First, you wouldn't normally need to implement Dispose() in a BO.  Do you have an unusual BO?

Are you using .NET 2.0?  According to the link you gave it seems the options are either upgrading to 2.0 or coding some memory management (UGH!) into this stuff you're writing.

RockfordLhotka replied on Monday, September 25, 2006

If you are implementing the full Dispose pattern, that would include implementing a finalizer. Objects with finalizers are reclaimed by the GC much slower than objects without a finalizer.

Which is a way of saying that you should avoid implementing the Dispose pattern if you can avoid it!

You should only implement Dispose/finalizer methods for objects that contain "expensive" resources - like file handles, synchronization primitives, database connections and things of that nature. And then only if those are instance-level fields.


The other issue you are running in to, and this is common across Windows and most other modern operating systems that use virtual memory, is the concept of a "working set". Processes are allowed a certain number of pages of memory they can use. A process can exceed that amount of memory, but the extra pages can be (and likely will be) written out to the page file. The working set is maintained in memory.

There are other thresholds too - both at the Windows and .NET levels.

The GC comes in two flavors - client and server. The client version is designed to minimize visible impact to the user, so it tries to only run when it needs to, and when the application is idle. It is also the case that the Windows memory management/paging subsystem flushes the working set in a way to minimize visible impact on the user. Hellen Custer's book on Windows NT (maybe it has been updated since, I don't know) covers this in some detail.

(as an aside, this mechanism came from OpenVMS on the VAX, and has changed little over time)

The point being, Windows and .NET are managing the memory on your behalf, following a set of rules. Understanding those rules can help you minimize the impact. The biggest thing you can do is avoid implementing Dispose/finalize on objects unless it is actually necessary.

mheere replied on Wednesday, September 27, 2006

Thanks for the quick reply Rockford!  After more than a day checking all sorts I found that the problem is probably with the Oracle driver I am using - vs.10.2.0.100.  This driver loses memory in the following scenario:

A business object has returned a DataSet to the client. Sending this DataSet directly back to the database seems to cause the leak!! I store the DataSet readymade as a BLOB in the database (for good reasons!! - since I may have manipulated and added lots of additional disparate data into the DataSet).

Wrong code (causes the memory leak):  'Reports' is the BusinessObject
DataSet ds = Reports.ReportRunGetOwnerInfo(EntityType, EntityCode).Results;    // retrieves the data
 Reports.SaveBlobReturnKey(ds);                                                                                       // saves the DataSet as a blob

Correct code (no leak)
DataSet ds = Reports.ReportRunGetOwnerInfo(EntityType, EntityCode).Results;
DataTable dtTmp = ds.Tables[0];
ds.Tables.Remove(dtTmp);
DataSet dsNew = new DataSet();
dsNew.Tables.Add(dtTmp);
Reports.SaveBlobReturnKey(dsNew);

Can you or anyone shine a light on what might be happening here?

Cheers

Marcel

 

ajj3085 replied on Monday, September 25, 2006

mheere:
However, when this BO goes out of scope (and you do the Dispose() etc.) you would expect the memory to have gone down to what it was before.


This isn't quite right, IIRC.  The grabage collector in .Net is very very lazy.  It usually won't do anything until you're almost out of memory, which may take quite a while because of virtual memory.  Its lazy because collecting garbage is an expensive operation which would degrade the performance of your (and all) .Net applications.

mheere:
However, what happens here is that I am having to repeatedly call a BO which therefore causes my app to run eventually out of memory.

Are you actually getting an out of memory exception?

mheere:
A remarkable thing to note is that if you minimise and then restore the app it will bring the memory right down again - and so it continues.

The amazing thing is that this seems to apply to all Windows apps. 

This is true of all windows applications and as far as I know is by design.  The amount of memory actually doesn't change; the amount of working memory does.

mheere:
Start Word, type away, check the Task Manager's memory consumption for this app, now minimise/restore the app and the memory will have come right down.

Ahhh!!  Task Manager is NOT a valid way to determine how much memory an application is using.  This is well documented.

mheere:
This behaviour seems to be known (http://www.dotnet247.com/247reference/msgs/35/178959.aspx) but what I don't understand is that this phenomenon is killing my app so this MUST also happen to other people!

If your application is running out of memory, its likely because somewhere, somehow you're not disposing of IDisposable objects properly, and / or you aren't releasing all references to objects (Csla based or not) you are using.  Check with a fine toothed comb.

HTH
Andy

Copyright (c) Marimer LLC