Background Loader - Alternative to Lazy

Background

.NET has the Lazy class. Its purpose is to avoid loading large objects the first moment the application is loaded. It has benefits in memory usage and load-speed.

Well... the load speed is good, but then the user must wait everytime he requests an item for the first time.

Proposed Solution

The proposed solution allows for fast load-times and tries to avoid the wait at first access by pre-loading values in the background. However it does not help reduce memory usage as it will load all items, even those that may never be used.

I don't consider that to be a problem, specially if the purpose is to make the application responsive. After all, the Lazy pattern does not allow an item that was used only once to be collected.

How It Works?

The idea is simple. When a BackgroundLoader class is created, it puts itself into a list of "need to load" items, and signals the loader thread to run.

The loader thread is started at Lowest priority, so it will only run when the application is idle. While it has items to process, it continues to load them and, when there are no more items, it waits again.

OK... there's more. It changes itself to normal priority while loading items, so in case the items hold locks while loading, it does not risk acquiring that lock and then become inactive for its low priority.

Also, as it happens with lazy, when the value is requested, if it is not loaded yet, then it is loaded immediately.

The code:

using System.Collections.Generic;
using System.Linq;
using System.Threading;

namespace Pfz.Threading
{
  /// <summary>
  /// Class similar to Lazy. But instead of only loading values when they are first accessed, it tries to load the
  /// value when the application is idle. Will load immediatelly if the value is requested.
  /// You must instantiate the generic version.
  /// </summary>
  public abstract class BackgroundLoader
  {
    private static readonly ManagedAutoResetEvent _event = new ManagedAutoResetEvent();
    internal static readonly HashSet<BackgroundLoader> _items = new HashSet<BackgroundLoader>();

    static BackgroundLoader()
    {
      Thread thread = new Thread(_Run);
      thread.Name = "Background Loader";
      thread.Start();
    }
    private static void _Run()
    {
      var currentThread = Thread.CurrentThread;
      while(true)
      {
        currentThread.IsBackground = true;
        currentThread.Priority = ThreadPriority.Lowest;
        _event.WaitOne();

        while(true)
        {
          while(true)
          {
            // Keeps yielding while there are other threads wanting to run.
            if (!Thread.Yield())
              break;
          }

          currentThread.Priority = ThreadPriority.Normal;
          currentThread.IsBackground = false;

          BackgroundLoader item;

          lock(_items)
          {
            item = _items.FirstOrDefault();

            if (item == null)
              break;

            _items.Remove(item);
          }

          try
          {
            item._CreateValue();
          }
          catch
          {
            // Ignore any exceptions here.
          }

          currentThread.IsBackground = true;
          currentThread.Priority = ThreadPriority.Lowest;
        }
      }
    }

    internal BackgroundLoader()
    {
      lock(_items)
        _items.Add(this);

      _event.Set();
    }

    internal abstract void _CreateValue();
  }

  /// <summary>
  /// Class similar to Lazy. But instead of only loading values when they are first accessed, it tries to load the
  /// value when the application is idle. Will load immediatelly if the value is requested.
  /// </summary>
  public sealed class BackgroundLoader<T>:
    BackgroundLoader
  where
    T: class, new()
  {
    private object _lock = new object();
    internal override void _CreateValue()
    {
      if (_value != null)
        return;

      var lockObject = _lock;
      if (lockObject == null)
        return;

      lock(lockObject)
      {
        if (_value == null)
          _value = new T();

        _lock = null;
      }
    }

    private T _value;
    /// <summary>
    /// Gets the Value, creating it immediatelly if needed.
    /// </summary>
    public T Value
    {
      get
      {
        T result = _value;
        if (result != null)
          return result;

        lock(_items)
          _items.Remove(this);

        _CreateValue();
        return _value;
      }
    }
  }
}

And a sample (that uses Thread.Sleep instead of real code that does a slow loading):

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using Pfz.Threading;

namespace ConsoleApplication3
{
  internal sealed class LargeLoadTime
  {
    public LargeLoadTime()
    {
      Console.WriteLine("Creating LargeLoadTime instance.");
      Thread.Sleep(1000);
    }
  }
  internal static class Program
  {
    static void Main(string[] args)
    {
      bool canContinue = true;
      while(canContinue)
      {
        Console.Clear();
        Console.WriteLine("This very simple application will try to compare the speed of Lazy and");
        Console.WriteLine("Background loader. Lazy does not try to load items, even if the application is idle,");
        Console.WriteLine("So it will make the user wait when the item is needed.");
        Console.WriteLine();
        Console.WriteLine("1 - Uses the BackgroundLoader class");
        Console.WriteLine("2 - Uses the Lazy class");
        Console.WriteLine("Chose an option and press enter. Invalid options quit the program.");

        switch(Console.ReadLine())
        {
          case "1":
            _BackgroundLoader();
            break;

          case "2":
            _Lazy();
            break;

          default:
            canContinue = false;
            break;
        }
      }
    }

    private static void _BackgroundLoader()
    {
      Console.WriteLine("Creating five items that have a slow load time.");
      var list = new List<BackgroundLoader<LargeLoadTime>>();
      for(int i=0; i<5; i++)
        list.Add(new BackgroundLoader<LargeLoadTime>());

      Console.WriteLine("Now simulating the idle time of the application.");
      Console.WriteLine("In this 6 seconds wait, the slow loading items should be loaded.");

      Thread.Sleep(6000);

      Console.WriteLine("Now we will use the items with large load-time.");
      Stopwatch stopwatch = new Stopwatch();
      stopwatch.Start();

      foreach(var item in list)
        Console.WriteLine("Reading " + item.Value);

      stopwatch.Stop();
      Console.WriteLine("The total time was " + stopwatch.Elapsed);
      Console.WriteLine("Press Enter to return to main menu.");
      Console.ReadLine();
    }

    private static void _Lazy()
    {
      Console.WriteLine("Creating five items that have a slow load time.");
      var list = new List<Lazy<LargeLoadTime>>();
      for(int i=0; i<5; i++)
        list.Add(new Lazy<LargeLoadTime>());

      Console.WriteLine("Now simulating the idle time of the application.");
      Console.WriteLine("In this 6 seconds wait, the Lazy class will not load its items.");

      Thread.Sleep(6000);

      Console.WriteLine("Now we will use the items with large load-time. Unfortunately, we will wait now.");
      Stopwatch stopwatch = new Stopwatch();
      stopwatch.Start();

      foreach(var item in list)
        Console.WriteLine("Reading " + item.Value);

      stopwatch.Stop();
      Console.WriteLine("The total time was " + stopwatch.Elapsed);
      Console.WriteLine("Press Enter to return to main menu.");
      Console.ReadLine();
    }
  }
}