Recently one of our projects has had the need to send a large number of objects from Silverlight to .NET using CSLA. However we were having OutOfMemory Exceptions with the MobileFormatter. So about a week ago I sat down to try to improve the MobileFormatter a bit.
For those who are unfamiliar with how the MobileFormatter works the process is as follows:
1) The formatter takes a string and creates a XMLWriter from it.
2) The formatter converts all BOs to SerializationInfo DTOs. These are simple C# POCO objects that contain the name of each and every member in the class (fully qualified) and the values for those classes
3) The formatter hands these DTOs to the DataContractSerializer and serializes them to the XMLWriter.
At step three our program was running out of memory (it was using about 500MB in this serialization process).
The patch I'm including makes the following modifications to CSLA:
1) It allows the user to configure what class is used for mobile formatting. Now you can swap in a new class by simply changing a few lines of code.
2) I've created a modified MobileFormatter (called FACMobileFormatter.cs in the included patch). This formatter makes the following improvements upon the classic MobileFormatter:
a) All XML and DataContract dependancies are removed. Instead, SerializationInfo DTOs have Write and Read methods that dump their contents directly into or from a binary stream.
b) since we have large numbers of strings repeated over and over again in the data produced by the MobileFormatter, the FACMobileFormatter "interns" strings.
So instead of writing data like this:
"MyObj.First" "John" "MyObj.Last" "Doe"
"MyObj.First" "Jane" "MyObj.Last" "Doe"
The new formatter writes data in this format:
"MyObj.First" "John" "MyObj.Last" "Doe"
1 "Jane" 3 4
So each unique string is only written once to the output stream. Each additional instance of that same string (compared by content, not by reference) is written as a single int referring to the index of the original string.
Over all we've seen about a 5x improvement in memory usage during serialization, and about a 4x improvement in the size of the serialized data.
Before where the application would break while serializing 76,000 objects in a single list, the new formatter runs without a hiccup.
----
I've tried to find a place to submit patches on this site, but I can't find it. So perhaps this is the place to start? If nothing else, I'd like my changes to allow custom formatters to make it into the CSLA codebase. That way I can feel safer about using this in production code without being worried that we'll have to keep patching each new release of CSLA.
This is timely, because the CSLA dev team has been discussing various options for a better serialization model.
Clearly Microsoft is moving away from platforms that support BinaryFormatter and NDCS, and that means MobileFormatter (or similar approaches) are about to become the primary serialization model for all apps (Silverlight, Windows Phone, WinRT - the future is all about the Silverlight subset of .NET).
Right now we're looking at options around a MobileFormatter update or replacement (probably just a direct update) as part of a 4.3 (or later) release.
Here is information about contributing to CSLA .NET: http://www.lhotka.net/cslanet/faq/How%20to%20contribute.ashx
If you'd like to join the dev team as a contributor email me directly (rocky at lhotka dot net) and we can discuss.
To follow up on this topic, Sergey is now working on changes somewhat similar to what you've outlined here. The approach isn't quite the same, but some of the basic ideas are similar.
Perhaps most importantly in some ways, is that core CSLA and the Silverlight WcfProxy/WcfPortal types will allow swapping in any ISerializationFormatter type as part of the change.
So even if you don't like our changes, as long as your serializer implements ISerializationFormatter (from Csla.Serialization) you should be able to configure core CSLA and/or the data portal to use your serializer.
That should work great, as long as ISerializationFormatter completely replaces the MobileFormatter. I had some issues modifying the older code, since the Silverlight code had direct references to MobileFormatter sprinkled through the code. As long as I can completely override that though (even in cases of cloning), these changes should work fine.
Yes, that's the plan.
Most of the SL code uses the serialization factory and ISerializationFormatter. I think it is just the WcfProxy/WcfPortal that have hard-coded use of MobileFormatter.
In any case, we'll end up with two config settings - one for core CSLA (undo/clone), and one for the WCF data portal channel. That largely mirrors the behavior on the .NET side, where core CSLA and the data portal usually use different serializers.
Hi Sergey,
I noticed some strong activity in \Csla\Serialization folder.
Is it finished? Is it too soon to ask for directions concerning "Config setting for WcfPortal / Proxy which formatter to use and update WcfPortal under SL and proxy to use new setting." as planned in http://www.lhotka.net/cslabugs/edit_bug.aspx?id=984?
Copyright (c) Marimer LLC