.NET Collections Performance Test in WindowsXP and WindowsCE

• Download source - 20.05 KB

Introduction

In the article, we will test the performance of collections. In our project, the Executer will be created by ExecuterFactory and we have more than 18 types of derived classes. We get expensive cost with create Executer when running, so we need a solution to improve performance.

The best solution is Object Pool Pattern or Caching Pattern, the two patterns are different when the object state is be reset or not. The definition of Object Pool Pattern needs to reset the state (like member value) when getting an object from pool, but the Caching Pattern always keeps the state.

And I will develop an index cache.

Background

I will try the best .NET collection with my requirement. First, Queue(FIFO) and Stack(FILO) can't match my requirement, I need to keep the object in pool or cache and I will use a key to get the object. At last, I will try List(ArrayList), Dictionary(Hashtable), SortedList.
I will try List because my collection will not include too many elements, and in this requirement, I will know the key, so I think that the List(ArrayList) is the best.
But I need a testing report to convince a colleague.

What I Did

First, I tried to test how much cost will be lost for the object is be created by Factory Pattern.

static void TestNew()
{
    for (int i = 0; i < max; i++)
    {
        MyTest orz1 = new MyTest01();
        orz1.Id = i;
        MyTest orz2 = new MyTest02();
        orz2.Id = i;
        MyTest orz3 = new MyTest03();
        orz3.Id = i;
    }
}

static void TestFactory()
{
    for (int i = 0; i < max; i++)
    {
        int type = ((i % maxclass) + 1);
        MyTest orz1 = MyTestFactory.Create(type);
        orz1.Id = i;
        MyTest orz2 = MyTestFactory.Create(type);
        orz2.Id = i;
        MyTest orz3 = MyTestFactory.Create(type);
        orz3.Id = i;
    }
}		

The MyTest class is a simple class, and the MyClass01~20 is the derived class of MyClass.
And the MyTestFactory.Create is a Factory Method to create MyClass01~20.

I will add all classes to collection, then get class from collection.

static void TestDictionary()
{
    Dictionary<int, MyTest> table = new Dictionary<int, MyTest>();
    for (int i = 1; i < maxclass + 1; i++)
    {
        table.Add(i, MyTestFactory.Create(i));
    }
    for (int i = 0; i < max; i++)
    {
        MyTest orz = table[(i % maxclass) + 1];
        orz.Id = i;
        MyTest orz2 = table[(i % maxclass) + 1];
        orz2.Id = i;
        MyTest orz3 = table[(i % maxclass) + 1];
        orz3.Id = i;
    }
}

static void TestHashTable()
{
    Hashtable table = new Hashtable();
    for (int i = 1; i < maxclass + 1; i++)
    {
        table.Add(i, MyTestFactory.Create(i));
    }
    for (int i = 0; i < max; i++)
    {
        MyTest orz = table[(i % maxclass) + 1] as MyTest;
        orz.Id = i;
        MyTest orz2 = table[(i % maxclass) + 1] as MyTest;
        orz2.Id = i;
        MyTest orz3 = table[(i % maxclass) + 1] as MyTest;
        orz3.Id = i;
    }
}

static void TestArrayList()
{
    ArrayList list = new ArrayList();
    for (int i = 1; i < maxclass + 1; i++)
    {
        list.Add(MyTestFactory.Create(i));
    }
    for (int i = 0; i < max; i++)
    {
        MyTest orz = list[(i % maxclass)] as MyTest;
        orz.Id = i;
        MyTest orz2 = list[(i % maxclass)] as MyTest;
        orz2.Id = i;
        MyTest orz3 = list[(i % maxclass)] as MyTest;
        orz3.Id = i;
    }
}

static void TestList()
{
    List<MyTest> list = new List<MyTest>();
    for (int i = 1; i < maxclass + 1; i++)
    {
        list.Add(MyTestFactory.Create(i));
    }
    for (int i = 0; i < max; i++)
    {
        MyTest orz = list[(i % maxclass)];
        orz.Id = i;
        MyTest orz2 = list[(i % maxclass)];
        orz2.Id = i;
        MyTest orz3 = list[(i % maxclass)];
        orz3.Id = i;
    }
}

static void TestSortedList()
{
    SortedList<int, MyTest> slist = new SortedList<int, MyTest>();
    for (int i = 1; i < maxclass + 1; i++)
    {
        slist.Add(i, MyTestFactory.Create(i));
    }
    for (int i = 0; i < max; i++)
    {
        MyTest orz = slist[(i % maxclass) + 1];
        orz.Id = i;
        MyTest orz2 = slist[(i % maxclass) + 1];
        orz2.Id = i;
        MyTest orz3 = slist[(i % maxclass) + 1];
        orz3.Id = i;
    }
}

Test Result

Finally, I tested this code in WindowsXP, WindowsCE 5.0 Emulator and our WindowsCE 6.0 device.

=======Test in WindowsXP========
Class number: 20
Performance test with run 1000000time
Elapsed = 00:00:00.0727499 in [Void TestNew()]
Elapsed = 00:00:00.2222732 in [Void TestFactory()]
Elapsed = 00:00:00.0458594 in [Void TestArrayList()]
Elapsed = 00:00:00.0467893 in [Void TestList()]
Elapsed = 00:00:00.2287626 in [Void TestSortedList()]
Elapsed = 00:00:00.1790526 in [Void TestHashTable()]
Elapsed = 00:00:00.1090860 in [Void TestDictionary()]

=======Test in WindowsXP========
Class number: 20
Performance test with run 1000000time
Elapsed = 00:00:00.1094648 in [Void TestNew()]
Elapsed = 00:00:00.2111686 in [Void TestFactory()]
Elapsed = 00:00:00.0535788 in [Void TestArrayList()]
Elapsed = 00:00:00.0486265 in [Void TestList()]
Elapsed = 00:00:00.2336329 in [Void TestSortedList()]
Elapsed = 00:00:00.1785616 in [Void TestHashTable()]
Elapsed = 00:00:00.1092564 in [Void TestDictionary()]

=======Test in WindowsCE 6.0 Device========
Class number: 20
Performance test with run 100000time
Elapsed = 00:00:00.3068472 in [Void TestNew()]
Elapsed = 00:00:00.6614232 in [Void TestFactory()]
Elapsed = 00:00:00.2436380 in [Void TestArrayList()]
Elapsed = 00:00:00.1280703 in [Void TestList()]
Elapsed = 00:00:00.8922895 in [Void TestSortedList()]
Elapsed = 00:00:01.3348975 in [Void TestHashTable()]
Elapsed = 00:00:00.9218634 in [Void TestDictionary()]

=======Test in WindowsCE 5.0 Emulator========
Class number: 20
Performance test with run 100000time
Elapsed = 00:00:00.2321512 in [Void TestNew()]
Elapsed = 00:00:00.7489057 in [Void TestFactory()]
Elapsed = 00:00:00.2709390 in [Void TestArrayList()]
Elapsed = 00:00:00.1322352 in [Void TestList()]
Elapsed = 00:00:00.7034211 in [Void TestSortedList()]
Elapsed = 00:00:01.2571927 in [Void TestHashTable()]
Elapsed = 00:00:00.7799420 in [Void TestDictionary()]

In the result, we will find these arguments:

1. Factory Pattern needs more cost.
2. Generic collection is faster than normal collection.
3. But in XP, sometimes the List is poorer than ArrayList.

Reference

/// <summary>
/// Performance Counter to calculate function executing time.
/// </summary>
public class Performance
{
    public delegate void CalculateHanlder();

    /// <summary>
    /// Using Performance.CalculFunction(MethodName)
    /// It will print executing time in Console.
    /// </summary>
    /// <param name="method">A none argument method</param>
    public static void CalculateMethod(CalculateHanlder method)
    {
        Stopwatch st = new Stopwatch();
        st.Start();
        method.Invoke();
        st.Stop();
        string msg = string.Format("Elapsed = {0} in [{1}]", 
			st.Elapsed.ToString(), method.Method.ToString());
        Console.WriteLine(msg);
        Debug.WriteLine(msg);
    }
}

Performance.CalculateMethod(TestNew); 

I use this code to calculate the executing time of method. The CalculateHanlder is a delegate, pass a none argument method to calculate time.

IndexCache

public interface IIndexCache
{
    void Register(object obj);
    void Unregister(object obj);
    object New(int index);
    void Delete(int index);
}

public interface IIndexCache<T>
{
    void Register(T obj);
    void Unregister(T obj);
    T New(int index);
    void Delete(int index);
} 

First, declare a normal and a generic interface named IIndexCache.

• Register: Implement this function to register an object into cache.
• Unregister: Implement this function to unregister an object from cache.
• New: Implement this function to get an object from cache.
• Delete: Implement this function to put back an object in cache.

public abstract class _IndexCache
{
    protected enum ObjectState
    {
        NO_USE,
        USING
    }

    protected List<ObjectState> m_objState = new List<ObjectState>();

    protected _IndexCache()
    {
    }
} 

I don't hope the object in cache needed to implement interface or inherit class, so I created an abstract class _IndexCache.

I declared an enum ObjectState, and use it to keep the state of object.

public class IndexCache :_IndexCache,IIndexCache
{
    /// <summary>
    /// Using for pool
    /// </summary>
    ArrayList m_Pool;

     /// <summary>
    /// Not use, set private to disable new.
    /// </summary>
    public IndexCache() : base()
    {
        m_Pool = new ArrayList();
    }

    #region IIndexCache Members

    /// <summary>
    /// Use IndexCache<X>.New(index) to get a instance.
    /// if the pool is empty or object is not using will return null.
    /// </summary>
    /// <param name="index">index in cache</param>
    /// <returns>T is a type of class</returns>
    public object New(int index)
    {
        object obj = null;
        if (index < m_Pool.Count && m_objState[index] == ObjectState.NO_USE)
        {
            obj = m_Pool[index];
            m_objState[index] = ObjectState.USING;
        }
        else if (m_objState[index] == ObjectState.USING)
        {
            throw new Exception("[" + index + "] was be used.");
        }
        else if (index >= m_Pool.Count)
        {
            throw new Exception("[" + index + "] not exist in cache.");
        }
        return obj;
    }

    /// <summary>
    /// Put back into cache
    /// </summary>
    /// <param name="index">object index in cache</param>
    public void Delete(int index)
    {
        if (index < m_Pool.Count)
        {
            m_objState[index] = ObjectState.NO_USE;
        }
        else
        {
            throw new Exception("[" + index + "] not exist in cache.");
        }
    }

    /// <summary>
    /// Register an object in cache
    /// </summary>
    /// <param name="obj">An object to register</param>
    public void Register(object obj)
    {
        m_Pool.Add(obj);
        m_objState.Add(ObjectState.NO_USE);
    }

    /// <summary>
    /// Unregister an object.
    /// </summary>
    /// <param name="obj">An object to unregister</param>
    public void Unregister(object obj)
    {
        int index = m_Pool.IndexOf(obj);
        m_Pool.Remove(obj);
        m_objState.RemoveAt(index);
    }

    #endregion

    /// <summary>
    /// Get the information for this cache.
    /// </summary>
    /// <returns>information</returns>
    public new string ToString()
    {
        return typeof(object).ToString() + 
		" IndexCache has [" + m_Pool.Count + "] items.";
    }
}

public class TIndexCache<T> : _IndexCache, IIndexCache<T>
{
    /// <summary>
    /// Using for pool
    /// </summary>
    List<T> m_Pool;

    /// <summary>
    /// Not use, set private to disable new.
    /// </summary>
    public TIndexCache()
        : base()
    {
        m_Pool = new List<T>();
    }

    /// <summary>
    /// Use IndexCache<X>.New(index) to get a instance.
    /// if the pool is empty or object is not using will return null.
    /// </summary>
    /// <param name="index">index in cache</param>
    /// <returns>T is a type of class</returns>
    public T New(int index)
    {
        T obj = default(T);
        if (index < m_Pool.Count && m_objState[index] == ObjectState.NO_USE)
        {
            obj = m_Pool[index];
            m_objState[index] = ObjectState.USING;
        }
        else if (m_objState[index] == ObjectState.USING)
        {
            throw new Exception("[" + index + "] was be used.");
        }
        else if (index >= m_Pool.Count)
        {
            throw new Exception("[" + index + "] not exist in cache.");
        }
        return obj;
    }

    /// <summary>
    /// Put back into cache
    /// </summary>
    /// <param name="index">object index in cache</param>
    public void Delete(int index)
    {
        if (index < m_Pool.Count)
        {
            m_objState[index] = ObjectState.NO_USE;
        }
        else
        {
            throw new Exception("[" + index + "] not exist in cache.");
        }
    }

    /// <summary>
    /// Register an object in cache
    /// </summary>
    /// <param name="obj">An object to register</param>
    public void Register(T obj)
    {
        m_Pool.Add(obj);
        m_objState.Add(ObjectState.NO_USE);
    }

    /// <summary>
    /// Unregister an object.
    /// </summary>
    /// <param name="obj">An object to unregister</param>
    public void Unregister(T obj)
    {
        int index = m_Pool.IndexOf(obj);
        m_Pool.Remove(obj);
        m_objState.RemoveAt(index);
    }

    /// <summary>
    /// Get the information for this cache.
    /// </summary>
    /// <returns>information</returns>
    public new string ToString()
    {
        return typeof(T).ToString() + " IndexCache has [" + m_Pool.Count + "] items.";
    }
} 

• Register: Invoke the function, it will add an object into cache, and will add state for this object
• UnRegister: Invoke the function, it will remove an object from cache, and will remove state for this object
• New: Invoke the function, it will return an object and change the object state
• Delete: Invoke the function, it will set the object state to NO_USE

History

• v1.1 Add IndexCache
• v1.0 New article