Csla data consolidation advice

kids_pro posted on Wednesday, August 23, 2006

Dear All,

I plan to migrate my vb6 app to .net using Csla + Sql Server 2000 (C#).
My application will be deploy in different region where each region is not connected (LAN/WAN) to the central server (Even if region can contact to the central server through dial-up connection but it to expensive to do data entry this way). 

On monthly basis each region need to submit their update data to the central server for consolidate. I am not sure if anyone had the same experience?

This is what I want to implement:

1. I am going to do a serialization of EditableRootCollection (this will include in child/grand child) and save it to a file(binary format and wrap it inside .zip to save space when send file through slow connection e-mail)

2. When the data received at the central server user will upload the file. I will do deserialization and verify if the data exist update otherwise insert.

Does this process make sense or someone has a better approach?

Thanks,
kids

tiago replied on Thursday, August 24, 2006

kids_pro:

2. When the data received at the central server user will upload the file. I will do deserialization and verify if the data exist update otherwise insert.

How would the central server know if the data already exists?

Your BOs do have an ID. Correct?

That's what the central server will be using to check for existence. Correct?

In that case, I think you must generate the ID locally. Your problem is a replication problem (there must be a pattern for this but I don't know the name).

I know 2 techniques to solve the replication problem:

A) Use GUID instead of integer autoincrement. The drawback is that GUIDs are bigger than INTs. Your tables will be bigger and your application will be a bit slower.

B) Use manual incremented integers. Example for 9 sites plus central site (total 10)
site 1 - starts with ID=1 and increments by 10 (1,11,21,31,...)
site 2 - starts with ID=2 and increments by 10 (2,12,22,32,...)
...
site 9 - starts with ID=9 and increments by 10 (9,19,29,39,...)
central server - starts with ID=10 and increments by 10 (10,20,30,40,...)

As you can see, ID will never overlap. Of course you should leave room for expansion so for 10 sites I would use at least 15 or 20 increments, even if my client tells me he's not planning to expand. Any way you could migrate to bigger increments later on. The problem would be to correct all foreign key IDs. But it's feasible.

kids_pro replied on Thursday, August 24, 2006

Tiago,

I do not have any problem with ID & existing check.
I would love to know the pattern that you mention.
Since I just take my step into serialize & deserialize in cosolidation process (not yet implement but I think it would work in theory).

RockfordLhotka replied on Thursday, August 24, 2006

kids_pro:

1. I am going to do a serialization of EditableRootCollection (this will include in child/grand child) and save it to a file(binary format and wrap it inside .zip to save space when send file through slow connection e-mail)

2. When the data received at the central server user will upload the file. I will do deserialization and verify if the data exist update otherwise insert.

Does this process make sense or someone has a better approach?

I would not take this approach. Business objects are not DTOs (data transfer objects), and what you need is a data transfer object - you are transferring data after all.

But first, you should consider that data replication is a very old and well understood problem. There are some very good solutions in the database world for data replication. I would first look at data replication solutions (like DTS) before writing your own replication by hand.

If you decide to write your own replication by hand, I would export the data into the most efficient format possible. That is NOT a serialized object. It is probably a binary format - but that may be too difficult to maintain. The next most efficient is a fixed-length text file, but that may be too restrictive. The next most efficient is a tab-delimited or comma-delimited file, which is relatively easy to maintain and is not overly restrictive. That'd be my first choice, followed by the far less efficient, but even easier, XML file. Then you transfer the file to the server, and have a process that reads the file and puts the data into the database.

The process of exporting the data can be implemented by CSLA objects. That is a use case after all. Similarly, the process of importing the data can be implemented by CSLA objects - it is also a use case (a different one obviously).