2

This is a general architecture question, hopefully to folks out there already using EF in final applications.

We have a typical N-Tier application:

  • WPF Client
  • WCF Services
  • EF STE DTO's
  • EF Data Layer

The application loads all known business types during load time (at the same time as the user logs in) then loads a very large "Work Batch" on demand, this batch is around 4-8Mg and is composed of over 1.000 business objects. When we finish loading this "Batch" we then link everything with the previously loaded business types, etc...

In the end we have around 2K-5K business objects in memory all correctly reference so we can use and abuse LINQ on the client side, we also do some complex math on all these objects on the client side, so we really need the large graph.

The issue comes when we want to save changes to the Database. With such a large object graph, we hardly want to send over everything again through the Network.

Our current aproach, which I dislike, given the complexity of the T4 templates so far, is to detach and attach everything on update. We basically want to update a given object, detach it from the rest of the graph, send it over the network, updated it on the WCF side, and then reattach it again on the client side. The main problem is when you want to update linked objects, let's say you add something that has a reference for something that is also added, then another reference to something modified, etc. This forces a lot of client code to make sure we don't break anything.

All this is done with generated code, so we are talking about 200-800 lines of T4 code per template.

What I'm looking at right now is a way to customize serialization and deserialization of the STE's, so that I can control what is sent over the network or not, and be able to update batches instead of just a single STE. Checking references, see if those references are Unchanged or not; if not don't serialize, if yes serialize and update everything just by attaching it to the context on the WCF side.

After some studying I found 2 solutions to this method.

One is by writing a custom DataContractSerializer.

The second one is by changing the STE template created by EF and playing around with the KnownTypeAttribute, instead of generating it for each reference type, have it reference a method that inspects the object and only marks for serialization references that are not unchanged.

  • Has anyone ever come across this issue before?
  • What solutions did you use?
  • What problems did you encounter down the line?
  • How easy was it to maintain the templates created?
Ladislav Mrnka
  • 360,892
  • 59
  • 660
  • 670
David Rodrigues
  • 622
  • 5
  • 8

1 Answers1

0

I don't know whole application design but if you generally load the work batch to the service and then send it to the client to play with it, it looks like service layer is somehow unnecessary and you can directly load data from database (and you will get much better performance). Depending on complexity of computation you can also do some computation directly in the database and you will again get much better performance.

Your approach to save only part of the graph is abuse to STE concept. STE works in manner - you load the graph, modify the graph and save the same graph. If you want to have a big dataset for reading and save only small chunks it is probably better to load data set for reading and once you decide to update a chunk, load only the chunk again, modify it and send it back.

Interfering the internal STEs behavior is imho the best way to lost some changes in some corner / unexpected scenarios.

Btw. this somehow looks like a scenario for syncing local database with a global one - I have never done that but it is quite common in smart-clients.

Ladislav Mrnka
  • 360,892
  • 59
  • 660
  • 670
  • Not really, because this is used over the Internet, and exposing the SQL Server directly is out of the question, Infrastructure rules, DMZ's, etc. prevent you from doing that. Not to mention that some people that will use this application are inside controled internets, and usually port 80 is open for HTTP, while TCP port 1433 to SQL Server is many times closed. – David Rodrigues May 10 '11 at 14:42
  • We can't only have chunks of the object graph, as some heavy math is applied to the entire graph, which updates a lot of it. The final result of that is what the user sees and perceives. We are also not getting the entire database, as it contains several of these large object graphs, we only load one at a time, as per user demand. The business is management of applications to european R&D funding, so we need to load the entire "project" do all the math of what needs to be cut, what are the actual costs, investments, what's elegible or not, what is actually going to be payed by the fund, etc. – David Rodrigues May 10 '11 at 14:45
  • Ok, now it makes sense with using WCF but still the idea with syncing local DB with global DB over WCF (it should be possible with MS Sync framework) sounds like solution with much less problems. I don't think that STEs are what you need because their purpose is different. – Ladislav Mrnka May 10 '11 at 15:08
  • Well, I looked into Sync when I was designing the first version of the Architecture, what made me back away from it was that you had to store a local copy of your data, either by implementing a local store or by deploying SQL Server Express (which would disable click once deployment), and also STE's give you some good WPF binding functionality out of the box, Many relationships are implemented as TrackableCollection which wraps around ObservableCollection, etc. Right now it works well, it's in production, and no data is lost. Problem is the templates look ugly, I think that exists a better way – David Rodrigues May 10 '11 at 15:19
  • Sync framework supports .NET compact as well and once you have local storage you can use EF directly with databinding as well. Templates look ugly, they are hard to develop, hard to maintain, hard to debug etc. and still nobody says that your change will work because one thing is to modify the object graph and one thing to keep it consistent with internal STE tracking. – Ladislav Mrnka May 10 '11 at 15:23
  • After further reading on Sync I found some constraints: EF 4.1 only works with SQLCE 4.0, EF 4.0 works with SQLCE 3.5 but doesn't support identity columns (which we have in most tables). Sync framework doesn't work with SQLCE 4.0. So basically to use Sync I'd have to use SQLCE 3.5 with EF 4.0 and would have to change all identity to GUIDs and handle the generation, along with changing all the code we have working now with STE's to Sync WCF Services. It's a lot of work! There's also security issues, we don't want to sync the entire Database, so would have to write more custom code there. – David Rodrigues May 10 '11 at 17:42
  • Ok, obviously some MS tools are not up to date with others. It was just idea. In the same time I don't think that your idea will be less work I even doubt that it will work but than are only my suspicion – Ladislav Mrnka May 10 '11 at 19:13