Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / database / SQL-Server

Using SQL Bulk Copy with your LINQ-to-Entities datacontext - Part 1

5.00/5 (1 vote)
27 Aug 2013CPOL 16.7K  
This tip shows how to significantly speed up inserts of large amounts of data using the entity framework.

Introduction

Linq-to-Entities is great for many things but it doesn't do very well with huge amounts of inserts. Extending the data context partial class with the following though will do the trick. All you need to do is to create your LINQ entities just as you usually do and call the BulkInsertAll method on the datacontext. Also, do not use this datacontext for anything else before doing the bulk insert. The connection string of the datacontext that we use to create the SQLConnection here loses its password after its first use for some reason and that makes it, obviously, a bit difficult to connect. This has also been posted on my blog. If you are using POCO objects and EF5 take a look at this

C#
public partial class MyEntities
{
    partial void OnContextCreated()
    {
        CommandTimeout = 5 * 60;
    }

    public void BulkInsertAll<t>(IEnumerable<t> entities)
    {
        entities = entities.ToArray();
            
        var ec = (EntityConnection) Connection; 
        var conn = (SqlConnection) ec.StoreConnection;
            
        conn.Open();
        Type t = typeof(T);

        var entityTypeAttribute = (EdmEntityTypeAttribute)t.GetCustomAttributes
        (typeof(EdmEntityTypeAttribute), false).Single();
        var bulkCopy = new SqlBulkCopy(conn) { DestinationTableName = entityTypeAttribute.Name };

        var properties = t.GetProperties().Where(EventTypeFilter).ToArray();
        var table = new DataTable();

        foreach (var property in properties)
        {
            Type propertyType = property.PropertyType;
            if (propertyType.IsGenericType &&
                propertyType.GetGenericTypeDefinition() == typeof(Nullable<>))
            {
                propertyType = Nullable.GetUnderlyingType(propertyType);
            }

            table.Columns.Add(new DataColumn(property.Name, propertyType));
            bulkCopy.ColumnMappings.Add(new SqlBulkCopyColumnMapping(property.Name, property.Name));
        }

        foreach (var entity in entities)
        {
            var e = entity;
            table.Rows.Add(properties.Select(property => 
            GetPropertyValue(property.GetValue(e, null))).ToArray());
        }

        bulkCopy.WriteToServer(table);
        conn.Close();
    }

    private bool EventTypeFilter(System.Reflection.PropertyInfo p)
    {
        var attribute = Attribute.GetCustomAttribute(p, 
        typeof(EdmScalarPropertyAttribute)) as EdmScalarPropertyAttribute;

        if (attribute != null) return true;

        return false;
    }

    private object GetPropertyValue(object o)
    {
        if (o == null)
            return DBNull.Value;
        return o;
    }
}

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)