Introduction
Persistent data has come a long way from plain files, data with indexes like Berkley DB or DBase, SQL servers with transactions, stored procedures and referential integrity. Last but not least, pure object oriented databases have been developed. XML files became very popular to store small pieces of information or do information interchange. In fact, there is no Swiss army knife or general purpose solution for persistent data. But for so called business applications, SQL and relational databases are still the first choice.
From the developers point of view, pure object oriented databases are eye candy. The code looks great and is not cluttered with database stuff. But for the stakeholders, there are many disadvantages using OO databases. It begins by the (often) extortionate licensing costs and ends with the IT department who hasn't any idea how to maintain (backup, convert, etc.) such a database. They object that mature and stable database systems are available. And some of them like PostgreSQL are free and open source. Some of them don't need hardware or maintenance like the Microsoft Azure cloud service.
In this area of conflict, the object relational mapping (ORM) frameworks came on the plan. ORM frameworks make relational databases look like object oriented (or object relational) ones. However the data is stored in a relational (or even simpler) database.
Why Should We Use a ORM Framework in General?
First of all, the ideal project for a relational database (and therefore for ORM) is a complex business application with a lot of logic (so called business logic) behind them. Typical business application domains are accounting and enterprise resource planning (ERP). Content management systems (CMS) for the web, organizers and business planners are such application domains too. Anything with complex relations and logic between the data (so called entities) fits in this concept. Maybe it is not the best idea to use relational databases or ORM if you need to store data that aren't structured or related to each other very much (like log files).
If you are familiar with SQL programming, you know that you always do the work twice while you develop such applications. Firstly you define your model on the database by creating a lot of tables and setting foreign keys between them. Then you write the code to access these tables, place queries and change values. Your logical entities are spread over several tables, you have to join them manually and maintain consistency (almost) manually. This have been the building block of the so called client/server applications since the middle of the 90s. It is implemented in the Microsoft .NET Framework as ADO.NET and deals with classes like DataSet, DataTable, DataRow and DataView.
Object Relational Mapping (ORM) came into play to access the database with objects. It acts as a layer between the programming language or environment on one hand and the underlying database on the other hand. As a result of this abstraction, the programming of business applications is much faster and the code is more maintainable. ORM Frameworks implement several ways to speed up the development process and increase maintainability. In general, they allow to access the database with objects that reflect entities in an (almost) natural way. Many of them allow to put LINQ queries against the database to get entities. And some of them decouple the business logic completely from the underlying database.
Two of the best known ORM frameworks are NHibernate (.NET) and the ADO.NET Entity Framework. Both of them have only a partial LINQ implementation and they are not known to be the fastest on the racetrack. In this article, I want to introduce DataObjects.Net from x-tensive. DataObjects.net is an ORM with a very high abstraction, an almost complete LINQ implementation and a really good performance.
Why is DataObjects.Net Worth to Give It a Try?
First of all, I gave it a try because of the automatic schema generation and maintenance. I've been programming SQL for years, and I'm completely fed up with this table, view and stored procedures declarations.
While working with it, I realized that this framework has solved almost all problems ORM comes along with. I can work with sessions and transactions in a transparent way and my entities look like real C#/VB objects. X-tensive have done a lot to reduce the chattiness (round-trips to the SQL server) to a minimum. For complex business applications, DataObjects.Net is even faster than working with the SQL server directly or using any lightweight ORM framework due to the optimizations they have done.
Feature and performance comparison with other ORM frameworks can be found on: ormbattle.net.
How Does DataObjects.Net Work?
DataObjects.Net uses aspect oriented programming (AOP) to inject the persistence and business logic in normal C#/VB classes. Aspects can be seen as some kind of function or delegate called at the beginning or the end of other functions. They don't need to be called in source code, they are injected after compiling the assembly in so call Intermediate Language (IL). DataObjects.Net uses the well known PostSharp framework (a framework for aspect oriented programming) to inject the mapping code. The model is declared in the form of simple classes with some attributes as shown in the following samples. It makes DataObjects.Net very elegant to use and it keeps all mapping stuff away from the user.
Download and Installation of DataObjects.Net
Before you can test drive the framework, you have to download and install it from the x-tensive homepage. In this article, I use Version 4.3 for Visual Studio 2010.
Behind the Scenes: DataObjects.Net and Visual Studio
You may ask yourself how PostSharp and DataObjects.Net is integrated into a Visual Studio solution or C# project. Due to the fact that PostSharp has to manipulate the compiled and linked assemblies, there must be a special build step after compilation. If you open your .csproj file in your Visual Studio solution you find an additional "import project" line:
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<Import Project="$(DataObjectsDotNetPath)\Common\DataObjects.Net.targets" />
This include does the magic behind the scenes. BTW: If you click right on your project in the Project Explorer window, you find "Unload Project". After you have unloaded your project, you find "Edit Project" to edit your project file. After editing, you can load your project again.
You find further information in the Manual under "Using DataObjects.Net in your projects and solutions".
Building Blocks
Every ORM framework maps some classes or objects against some data in a database. However every ORM framework has its own way to accomplish this. In the following sections, I will explain the building blocks of a DataObjects.Net application and put a glance at the ideas behind.
Domains and Data Storage
DataObjects.Net needs access to some kind of data storage, commonly an SQL server. The glue between the database and the C# objects is the so called Domain. The term domain has nothing to do with Internet domains, but comes from the so called Domain Driven Design (DDD). In technical terms, the Domain is the connector to the database. A Domain needs two things: First of all, a connection string to a database or data storage in general. All data will be stored there. Second, assemblies and namespaces where your persistent classes are defined. These classes are the so called 'Model'. The needed tables, indices, primary and foreign keys are created from this 'Model'. A DataObjects.Net Domain can be configured in the App.config file or in code. Here is a sample how this is done in code:
var config = new DomainConfiguration("sqlserver://localhost\\SQLExpress/DO40-Tests");
config.UpgradeMode = DomainUpgradeMode.Recreate;
config.Types.Register(typeof(SimpleEntity).Assembly, typeof(SimpleEntity).Namespace);
var domain = Domain.Build(config);
We have a connection string to a Microsoft SQL Server Express database named "DO40-Tests". Keep in mind that DataObjects.Net can generate tables and so on, but it can't generate any database itself. You must create the database "DO40-Tests" and ensure that DataObjects.Net can access it under the given connection string. After that, DataObjects.Net will do the rest.
Model
Persistent data is declared and accessed in form of C# classes. Persistent types must derive from the class Entity or some subclass of Entity. Persistent Fields are declared as properties with the [Field] attribute. In an inheritance hierarchy, the first or root class must be marked with the [HierarchyRoot] Attribute and at least one [Field] property must be declared with the [Key] attribute to become primary key. Here we have a simple entity with an int key and a string data (payload) field:
[Serializable]
[HierarchyRoot]
public class SimpleEntity : Entity
{
[Field, Key]
public int Id { get; private set; }
[Field(Length = 100)]
public string Data { get; set; }
}
These SimpleEntity objects aren't related in any way. The HierarchyRoot attribute denotes this class as a base for a hierarchy. Hierarchy in this context means base class for inheritance in the object oriented meaning. The next example shows how classes are derived from this base class:
[Serializable]
[HierarchyRoot]
public class BaseEntity : Entity
{
[Field, Key]
public int Id { get; private set; }
[Field(Length = 100)]
public string BaseEntityData { get; set; }
}
[Serializable]
public class DerivedEntity : BaseEntity
{
[Field(Length = 100)]
public string DerivedEntityData { get; set; }
}
As shown, the derived class inherits the normal C# way from the base class. It inherits all base properties and the primary key from the base class. Your can make an educated guess that all the casting stuff base <-> derived will work the normal C# way. Working with these persistent objects is almost the same as working with normal objects. In the next section, a typical parent/child or sometimes called master/detail relation is modelled:
[Serializable]
[HierarchyRoot]
public class ParentEntity : Entity
{
[Field, Key]
public int Id { get; private set; }
[Field]
public EntitySet<ChildEntity> Childs { get; private set; }
}
[Serializable]
[HierarchyRoot]
public class ChildEntity : Entity
{
public ChildEntity( ParentEntity parent )
{
Parent = parent;
}
[Field, Key]
public long Id { get; private set; }
[Field]
[Association(PairTo = "Childs", OnTargetRemove = OnRemoveAction.Cascade)]
public ParentEntity Parent { get; private set; }
}
The ParentEntity class is almost straight forward. In the ChildEntity, we see a ParentEntity 'Parent' field, where a reference to the parent is stored. And we see some business logic in the form of an Association attribute. In this case, the association pairs the field 'Parent' and the EntitiySet 'Childs'. Every ChildEntity which has a particular parent is automatically in the Childs EntitySet and vice versa. With the 'OnTargetRemove = OnRemoveAction.Cascade' action, childs are automatically deleted when a parent is deleted. To explain all the business rule features of DataObjects.Net goes far behind the scope of this introduction. But keep in mind that those things are possible with DataObjects.Net. Both classes are marked with the HierarchyRoot attribute because both classes are base classes (roots of an inheritance hierarchy).
In order to increase performance on queries, it is also possible to specify indices on some fields. They are declared with the Index attribute as shown in the next sample.
[Serializable]
[HierarchyRoot]
[Index("Data")]
public class IndexedDataEntity : Entity
{
[Field]
[Key]
public int Id { get; private set; }
[Field]
public int Data { get; set; }
}
Basic Operations
After setting up a Domain and declaring a Model, it is time to run some operations. Although DataObjects.Net allows a very natural way to access data, it fully supports sessions and transactions.
Sessions and Transactions, Creation and Destruction of Persistent Objects
The next sample shows how sessions and transactions are used programmatically. Especially for transactions, this can also be done in a declarative way using attributes.
using (Session.Open(domain))
{
using (var transactionScope = Transaction.Open())
{
new SimpleEntity { Data = "SimpleEntity 1" };
new SimpleEntity { Data = "SimpleEntity 2" };
transactionScope.Complete();
}
}
The use of the IDisposable pattern (using) guarantees that the session and transaction is closed in case of an exception. The SimpleEntity objects are really persisted in this sample without providing any transaction or session to them. To achieve this, the framework uses a hidden (thread static) current session and current transaction. The current session and current transaction are automatically set by the Open() methods. In the next sample, a ParentEntity and ChildEntity objects will be created. Due to the constructor of the ChildEntity class, it is impossible to create ChildEntity objects without a parent. DataObjects.Net supports the constructor semantic of normal C# classes.
using (Session.Open(domain))
{
using (TransactionScope transactionScope = Transaction.Open())
{
var parent = new ParentEntity();
new ChildEntity(parent);
new ChildEntity(parent);
new ChildEntity(parent);
transactionScope.Complete();
}
}
As shown, it is very easy to create persistent objects with DataObjects.Net. It is also no problem to remove or delete such an object. It is enough to call the Remove() member function on the object. We should bear in mind that these newly created objects will last forever or at least till the Remove() method is called. But when our local variables ran out of scope or our application is terminated, how can we get our objects again? The answer is simple: We use LINQ (language integrated query) to get our objects from the database. This is shown in the next section.
Querying Persistent Objects with LINQ
The starting point of every DataObjects.Net query is Query.All<>. The next query returns all DerivedEntity objects in an arbitrary order.
using (Session.Open(domain))
{
var linqQuery = from entity in Query.All<DerivedEntity>() select entity;
foreach (var entity in linqQuery)
{
}
}
Once we have an object, we can use the references and collections in the normal C# way. In the next example, we iterate over the Child collection in ParentEntity objects.
using (Session.Open(domain))
{
var linqQuery = from entity in Query.All<ParentEntity>() select entity;
foreach (var parentEntity in linqQuery)
{
foreach( var childEntity in parentEntity.Childs )
{
}
}
}
DataObjects.Net has an almost complete LINQ support. The query in the next sample is more complex. It puts a condition (where) on the query and sorts the result (orderby).
using (Session.Open(domain))
{
var linqQuery = from entity in Query.All<IndexedDataEntity>()
where entity.Data < 1000
orderby entity.Data
select entity;
foreach (var entity in linqQuery)
{
}
}
In fact, LINQ queries can be much more complex. But that goes far behind this introduction.
Conclusion
We use DataObjects.Net 4.0 and PostgreSQL in a new project with about 60 persistent classes and currently 10 million entities. At the beginning, we had some problems with the performance of the PostgreSQL server. But after configuring adequate cache size (the defaults are very low) and some minor optimizations, it is really fast and is comparable to the Microsoft SQL server in almost all cases. We are working with PostgreSQL 9.0 Beta4 64Bit. I am quite impressed how fast and easy database application development can be. To put it in a nutshell: DataObjects.net can be a good choice to speed up the development process especially for new projects.
