Reattaching Entity Graphs with the Entity Framework

• Download sample - 40.3 KB

Introduction

One of the main challenges when using the Entity Framework (version 1) is exchanging entity graphs between tiers of a Service Oriented Application.

When using Silverlight for building Rich Internet Applications, or Rich Client Applications with Windows Forms or WPF, it is natural to provide coarse grained services on your service layer. Consider, for instance, a 'Software-as-a-Service' invoicing application written in Silverlight… The server-side invoicing service could provide operations like the following:

• GetProducts(ProductSelectionCriteria)
• GetCustomers(CustomerSelectionCriteria)
• GetInvoices(InvoiceSelectionCriteria)
• GetSuppliers(SupplierSelectionCriteria)
• SaveProduct(Product)
• SaveCustomer(Customer)
• SaveInvoice(Invoice)
• SaveSupplier(Supplier)

These service operations allow you to implement the client with a minimum of 'post backs' to the server, which is one of the reasons we like to use Rich Internet Applications.

Unfortunately, we will see that the Entity Framework (version 1, at least) does not support this scenario very well.

Attached/Detached Entities

The Entity Framework, as most ORMs, operates on entities under its management. Whenever an EF query is executed (and MergeOption is not set to NoTracking), the instantiated entities are 'attached' to the EF context.

This means that whatever happens to those entities (their properties and relations are changed), the Entity Framework will be aware of it and will be able to save those changes.

However, when an entity is sent to a different tier, it leaves the Entity Framework context, and becomes 'detached'. Whatever changes are done to the entity in the detached state, no EF context will be aware of those changes, and will not be able to save them.

The solution sounds obvious, but is unfortunately not that simple: reattach the entity to an EF context before saving it…

Although this sounds easy, the implementation of the Entity Framework (version 1) does not allow us to attach an entity graph containing both new and changed entities. Basically, the ObjectContext class of the Entity Framework provides two methods to (re)attach entities:

void AddObject(string entitySetName, object entity)
void Attach(IEntityWithKey entity)

The first method, AddObject, is to be used exclusively to attach new instances to an Entity Framework context. For instance, if the invoicing application creates a brand new invoice, it would be able to attach it to a context using the AddObject method. Or not… as we'll see later.

The second method, Attach, is to be used exclusively for entities that already exist on the context, and only need to be attached. For instance, if the invoicing application retrieves an existing invoice, edits it, and then wants to save the changes, it would need to use the Attach method to attach the edited invoice before saving it.

Both methods, AddObject and Attach, are able to copy with object graphs, and, to some extent, even with object graphs containing a mix of new and attached entities.

However, none of those methods are able to handle a mix of attached and detached entities. And, this is a real issue if you understand that it means it is impossible to bring a set of two related detached entities in attached state, since as soon as one of those objects gets attached, you have a situation of an attached entity linked to an unattached entity.

The Entity Framework will not allow you to have an attached entity linked to a detached entity, even during transition phases. As a reaction, the Entity Framework will either throw an exception, or simply cut the link between your objects…

System.InvalidOperationException: The object cannot be added to the 
ObjectStateManager because it already has an EntityKey. 
Use ObjectContext.Attach to attach an object that has an existing key..

Basically, this means the Entity Framework cannot (re)attach to object graphs, and most scenarios where detached entities are to be reattached to be persisted by the Entity Framework will not work.

For instance:

• Creating a new invoice will not work: the new invoice instance will be linked to an existing (but detached) customer instance. The same will be true for the invoice lines, which will be linked to existing product instances. Calling ObjectContext.AddObject will result in an InvalidOperationException.
• Editing an existing invoice will not work (1): the invoice and invoice lines form a graph of objects to which detached and new instances can coexist, which is not supported by the Entity Framework.
• Editing an existing invoice will not work (2): suppose I edit an invoice by removing one of its lines, how will it be detected that an invoice line is to be deleted?
• Editing an existing invoice will not work (3): I edit and send back an invoice to be saved. The invoice is linked to a customer entity and to invoice line entities. The lines are linked to products, etc. To which extent should the object graph be persisted on the store? Should the relationship between invoice lines and products be persisted? Should the relationship between products and suppliers be persisted? The Entity Framework can guess which relations to persist and which not.

The EntityBag Solution

One solution developed by Danny Simmons (a Development Manager in the Data Programmability team at Microsoft) is the EntityBag (http://code.msdn.microsoft.com/entitybag/). The EntityBag is some kind of object context that can live on the client-side, and will store change tracking information. When the entity bag is sent to the server to be persisted, all changes can then be reproduced on an attached Entity Framework ObjectContext.

Although the EntityBag solves the issues discussed above, this solution also has some issues on its own. First, it requires you to run the Entity Framework on the client side, and is therefore not interoperable with Java or other non-.NET languages. But mainly, you will need to review the signatures of your service operations, you may need more service operations, and, above all, I find the modified service operations not following the very nature of WCF's data contracts.

For instance, if you want your RIA client to retrieve a list of invoices, and then have the opportunity to save an edited invoice, you could define the following operations:

Invoice[] GetInvoices(int customerId)
void SaveInvoice(Invoice editedInvoice)

With the EntityBag solution, you'll have to make sure to obtain an EntityBag on the invoice before starting to edit it, so you'll need an additional operation and an additional postback:

EntityBag<Invoice> GetInvoiceEntityBag(int invoiceId)

The GetInvoices method could remain unchanged, but the SaveInvoice would be changed into:

void SaveInvoice(EntityBag<Invoice> editedInvoiceEntityBag)

Besides the fact that the use of the EntityBag affects our Service Contract, it also introduces the use of Generics, which is not in line with the nature of WCF contracts.

The AttachObjectGraph Solution

The solution I propose in this article, and of which I provide the code here, consists of a single AttachObjectGraph Extension Method on ObjectContext, which can attach object graphs, combining both new and detached instances to a parameterizable extend.

The idea is to have a single AttachObjectGraph method that is smart enough to know how the given object graph is to be attached so that it is usable in all common scenarios.

The AttachObjectGraph method is defined as follows:

public static T AttachObjectGraph<T>(this ObjectContext context,
              T entity, 
              params Expression<Func<T, object>>[] paths
)

The paths argument surely seems awkward. But, as you will see, it provides a very handy and safe way of defining the paths of the graph to be attached.

To demonstrate the use of this method, I will use the following sample model, which is also available for download:

It's a typical example, but it's an example where most common scenarios come along. The scenarios we can test using this model include:

• creating a new invoice
• editing an invoice by changing properties (i.e., InvoiceDate, StateCode, Comments, …)
• editing an invoice by adding lines
• editing an invoice by editing lines (i.e., setting a different Quantity or Product)
• editing an invoice by removing lines
• changing the supplier of a product

All but the last of these scenarios will be supported by one single implementation of a SaveInvoice (server) operation:

void SaveInvoice(Invoice invoice)
{
    using (var context = new SampleInvoiceContext())
    {
        context.AttachObjectGraph(
            invoice,
            p => p.Customer,
            p => p.Lines.First().Product
        );
 
        context.SaveChanges();
    }
}

The AttachObjectGraph method receives as first argument the (root) entity to attach. In this case, an invoice. The remainder of the arguments (a params array of expressions) represent paths of relations to be included in the attach operation.

In the SaveInvoice operation above, we request to attach an invoice along with (its relation to) its customer, (its relation to) its invoice lines, and their related products. As the Lines property results in a collection, we can't immediately dereference its relational properties. Therefore, a call to the First() method is done. In reality, all lines will be attached. The call to First() is there only to provide code completion support in the editor and compile checking. Alternatively, the call could – if a matching overload would have been provided – be written as follows, in which case, there would be no compile time checking:

context.AttachObjectGraph(
    invoice,
    "Customer",
    "Lines.Product"
);

The SaveInvoice operation will thus attach its given invoice, including its customer, invoice lines, and products. It will, however, not touch the relation a product has with a supplier.

The AttachObjectGraph method solves the following issues:

• It can attach object graphs containing mixes of detached but existing entities and new entities, and hence solves our main issue.
• It does not add specific requirements to the Service Contract, and therefore can be applied transparently and fully in the spirit of WCF contracts.
• It does not require the Entity Framework to run on the client, and hence is not tied to .NET languages. It does, however, require the Entity Framework EntityKey property to roundtrip, which is done automatically by WCF, and can be hidden through the ExtensionDataObject property.
• It is secured to some level, as you have the guarantee that only entities in the given paths can be manipulated. Additionally, you could perform additional security checks and (business) logic between the call to AttachObjectGraph and SaveChanges.

This means that the AttachObjectGraph method provides a really interesting and powerful way of using the Entity Framework over tiers. Still, with some additional features, the method will be usable in almost any standard case.

Performance

The first version of the AttachObjectGraph method performed lazy loading of relations. Therefore, reattaching an invoice with 10 lines would result in 13 queries being issued to the database (one to get the invoice, one to get the customer relation, one for the lines relation, and 10 more for the product relation of each line). The more lines the invoice had, the more queries were issued to the database.

The AttachObjectGraph method has now been enhanced to perform an automatic preload of the object graph, such that the objects are already in memory when the attach operation is performed. The automatic preload consists of a single query loading the whole stored object graph. The 13 queries are reduced to 1 single query.

Of course, the automatic preload only works for entities and relations already present on the data store. When attaching a newly created invoice, no automatic preload can occur, and several queries will be issued to retrieve the customer and products the invoice is related to.

In most cases, entity graphs are created less often than they are used or manipulated. The automatic preload will therefore increase performance in most cases, while have no extra cost in most other cases.

Although the automatic preload built in the AttachObjectGraph method cannot perform for object graphs where the root entity is new, you could, yourself, knowing the specific situation, perform a 'manual' preload by simply querying the related existing entities in the context, prior to calling the AttachObjectGraph method.

The following code, for instance, 'manually' preloads all products:

var allproducts = from p in context.ProductSet select p;
allproducts.ToArray();

Additional Features

Attaching Collections

The SaveInvoice operation described earlier saves a single invoice. You might also be in a situation where you want to attach a collection of objects to the context. For instance, you might want to save a collection of products. Although you could loop over all products and attach them one by one using the AttachObjectGraph method, this would have a performance penalty as an automatic preload query would be issued for each item of your collection.

Therefore, I also provided an AttachObjectGraphs method, taking a collection of root entities as the first argument. Using this method, only one single preload query is executed, which would preload all of the products to be attached.

context.AttachObjectGraphs(
    products,
    p => p.Supplier
);

The method returns an array of attached root instances.

Not Updating Path Ends

The call to AttachObjectGraph, as we saw it above, has the following issue:

context.AttachObjectGraph(
    invoice,
    p => p.Customer,
    p => p.Lines.First().Product
);

When attaching an invoice, it will keep the link with its customer, with its invoice lines, and will also keep the link between the lines and their products. This is meant to be.

However, as a 'side effect', any changes done on the customer entity or on the product entities will be applied, and eventually saved as well. Every object within the scope of the object graph will potentially be updated. This could be a serious security thread, and to my knowledge, is an issue also known to the EntityBag solution.

In the case of a 'SaveInvoice' service operation, we want to save the invoice, but not the products. We allow the user to change the price on an invoice line, but we don't want to allow the user to change the price on our product entity, on our products catalogue!

Basically, we need a way to indicate whether changes on the entities at the ends of the paths of the object graph are allowed or not.

To allow this differentiation, I introduced an extension method on entities (IEntityWithKey), which is called "WithoutUpdate()", and only serves as a marker within the object graph path expressions. The method is not to be invoked (and would fail if invoked), but is used by the parser of the expression path arguments to mark the end of the path as updatable or not.

context.AttachObjectGraph(
    invoice,
    p => p.Customer,
    p => p.Lines.First().Product.WithoutUpdate()
);

By extending the path to the products with the WithoutUpdate() method, the invoice lines will be linked to their products, but any changes done on the product entities will not be reflected in the attached invoice (and hence not saved to the database).

Deleting Removed Entities

When calling the SaveInvoice operation with an updated invoice, that invoice will only contain its actual invoice lines. The AttachObjectGraph method is smart enough to detect whether invoice lines have been removed, and will remove those lines from the Lines collection on the attached invoice entity.

However, an invoice line that is removed from its invoice violates the constraint that every invoice line is to have an invoice. When saving the object graph to the database, it will fail.

Ideally, the AttachObjectGraph would detect that it cut a mandatory association, and would automatically perform a delete of the orphan entity.

Unfortunately, I was unable to retrieve, from the Entity Framework, metadata information whether an association is mandatory. So, I've implemented an alternative solution where the owned entity is to be decorated with an AssociationEndBehaviorAttribute, marking the role, for which removed entities are to be deleted, as owned.

Using partial classes, this can perfectly be done without modifying the code generated by the EDM designer, as the attribute is not to be set on the association end property, but on the owner entity type.

To mark the Envoice.Lines role as 'owned' (meaning its values are owned), apply the attribute as follows:

[AssociationEndBehaviorAttribute("Lines", Owned=true)]
partial class Invoice {
}

The AttachObjectGraph method will check for this attribute while navigating the property paths, and will automatically delete each entity which has been removed from the association. Exactly what we needed…

The Code

The sample attached to this article contains all the code of the AttachObjectGraph and AttachObjectGraphs methods, as well as a few extras.

The sample, furthermore, consists of the Entity Framework model of invoicing, a SQL script to reconstruct the (SQL Server) database, and a test project containing Unit Tests demonstrating several scenarios and features of the AttachObjectGraph(s) methods.

Although I talk about the AttachObjectGraph method, its realization is, in reality, a set of methods and classes which would bring us too far to explain in detail here.

The sample contains two additional features which are not discussed in this article, but for which you will find information in the following two posts on my blog:

• Entity Framework – a concurrency manager
• Entity Framework – compile-safe includes

The Sample Solution

The sample solution consists of the following projects and artifacts:

I highlighted the most important items, which I'll shortly describe here:

• RecreateDatabase.sql: a SQL Server script that will recreate the whole sample database needed to run the sample tests.
• ObjectContextExtension.cs: this is the place where you'll find the AttachObjectGraph method.
• Model.edmx (and Model.Designer.cs): the Entity Model generated by the EDM designer.
• Model.cs: partial class extensions to the Entity Model.
• SaveInvoiceTests.cs: test methods showing different scenarios calling a SaveInvoice operation.

The TestProject has a dependency to PostSharp. You will need to install PostSharp (http://www.postsharp.org/) to run the solution.

The Test Methods

As to demonstrate several features, and test different scenarios, I didn't provide a 'sample application', but rather a sample set of Unit Tests.

This is the list of Unit Tests provided:

• The ListTests class contains 'tests' that write data from the database to the console, and are, in fact, only useful to see what is in the database.
• The SaveInvoiceTests class contains tests that demonstrate different scenarios to call a SaveInvoice method, as described in this article.
• The SaveProductTests class contains similar tests to demonstrate the usage of the SaveProduct method.

Let's take a look at one of these test methods:

/// <summary>
/// Test the editing and addition of lines on an invoice to be saved.
/// </summary>
[TestMethod]
[RollingBackTransaction]
public void ExtendInvoiceTest()
{
    Invoice inv = this.GetDetachedInvoice(5);
    InvoiceLine line;
 
    this.ShowInvoice(inv, "Original invoice:");
 
    Assert.AreEqual(1329.98m, inv.TotalPrice);
 
    // Update invoice:
    inv.Customer = this.Customers[3];
    inv.InvoiceDate = inv.InvoiceDate.AddDays(3);
 
    // Update existing line:
    line = inv.Lines.Where(p => p.ID == 8).First();
    line.Product = this.Products[5];
    line.UnitPrice = line.Product.SalePrice;
 
    // Add an extra lines:
    line = new InvoiceLine();
    line.Quantity = 5;
    line.Product = this.Products[6];
    line.UnitPrice = line.Product.SalePrice;
    inv.Lines.Add(line);
 
    // Add another extra line:
    ...
 
    Assert.AreEqual(340.93m, inv.TotalPrice);
 
    // Save invoice:
    inv = this.SaveInvoice(inv);
    this.ShowInvoice(inv, "Saved invoice:");
 
    // Retrieve invoice from store and check total price:
    Assert.AreEqual(340.93m, this.GetDetachedInvoice(5).TotalPrice);
}

The test method is decorated with a TestMethod attribute (of course) and a RollingBackTransaction attribute. The source code of this is provided in the project. The RollingBackTransaction attribute rolls back whatever the test has done on the database, ensuring the database changes are not persisted between tests.

In this test, we first retrieve an invoice, show it (on the console, just for debugging ease), assert its price matches the expected value, then we update the invoice by assigning it a different customer, changing the invoice date, the product, and the unit price of an invoice line, and add two extra invoice lines.

Then, the invoice price is verified to be as expected, before actually saving the invoice. To conclude, the invoice is again retrieved from the database, and its invoice price verified.

Conclusion

Despite the limitations of the Entity Framework to reattach detached entities, the features of .NET 3.5, including Extension Methods, LINQ, Lambda expression parsing, Aspect Oriented Programming, partial classes and methods... allow us to extend and enhance the existing frameworks seamlessly.

Article History

• 24^th February, 2009 - v2: Reviewed version; fixed database recreate script; enhanced performance in sample by using preload; explained preload in article
• 4^th February, 2009 - v1: Initial version