Virtual Cache

• Download Sop.zip - 832.7 KB

Introduction

Virtual Cache is a high performance, scalable data caching implementation using the SOP data framework. This implementation uses the disk to extend memory capacity, limited only by the underlying hard disk drive (HDD). SOP provides very efficient RAM & Disk virtualization which was a feature inherited by the VirtualCache API for using SOP as the backing store for the cached data.

SOP's new, unique method that fuses together key algorithms & services for data management within the Data Store brings high degree of performance & scalability, "in-memory" performance when data fits in RAM, and close to "in-memory" performance when managing large data sets, bigger than RAM capacity. Following describes few of the key unique data handling improvements of SOP:

• Application data container disk based B-Tree was purposely designed to work optimally with an internal caching mechanism utilizing Most Recently Used (MRU) algorithm
  • All operations occur in memory with zero data persistence when data fits in memory.
  • When data doesn't fit in memory, page swapping driven by the MRU unit between RAM & Disk occurs in a very efficient manner using Async, unbuffered I/O to produce minimal latency, very high speed data access & management.
• Data Block Management logic that encourages bulk I/O.
• Custom fitted (low-level) lightweight, Copy On Write (COW) based Transaction for data protection.
• Please check out http://sop.codeplex.com for more details of SOP standouts. :)
• Please visit this link for downloading the source code: https://sop.codeplex.com/releases/view/119752

Background

For .NET data caching, a little over two years ago, I initially implemented an Entity Framework 2^nd level data caching, extending this EF caching solution by Julie Lerman. This solution has been bundled with the SOP framework part of the open source package, in SOP v4.5 release. As time went by and few releases after, the .NET 4.5 framework offered a new, general purpose standalone object caching solution, the ObjectCache derived MemoryCache implementation.

MemoryCache is a great solution for caching data using the host computer's memory (RAM). However, the amount of data that can be cached is limited by the memory available to the application. In order to alleviate this limitation, I decided to create a solution that uses the host computer's HDD for extending the memory using the available storage space on the disk. The performance & low footprint bar is high for this endeavor since I don't want to duplicate existing data caching initiatives by Microsoft and 3^rd party solution providers. Microsoft's distributed caching, the AppFabric is already available and recommended for Enterprise class, multi-server caching.

However, for applications that don't benefit from distributed caching due to latencies (e.g. network latency, multi-AppFabric service orchestration, etc.) brought about by this solution, VirtualCache is now available. This is a solution optimized, designed from the ground up for virtualizing RAM and disk with minimal footprint, to offer the optimal data caching solution that fits the use-case scenario outside of what AppFabric is recommended for.

Using the Code

Please check out .NET 4.5 MemoryCache documentation & sample discussion for details on using the ObjectCache API, which is the abstract base class of (MemoryCache &) VirtualCache, in this link: http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache(v=vs.110).aspx.

Sample programs specifically for the VirtualCache implementation are available for download. The "VirtualCache" & "VirtualCacheThreaded" sample programs in the SOP v4.7 zip file (attached or downloadable from http://sop.codeplex.com) show a basic usage of this highly optimized cache virtualizer.

Also, below is a snippet to show the basic usage of the VirtualCache API:

using System;
using System.Runtime.Caching;

namespace Sop.Samples
{
    /// <summary>
    /// Virtual Cache demo module.
    /// </summary>
    public class VirtualCacheDemo
    {
        private Log.Logger _logger;
        public void Run()
        {
            using (Sop.Caching.VirtualCache vc = new Sop.Caching.VirtualCache("MyCacheStore"))
            {
                _logger = vc.Logger;
                _logger.LogLevel = Log.LogLevels.Verbose;
                _logger.Information("Start of VirtualCache demo.");
                Populate(vc);
                RetrieveAll(vc);
                _logger.Information("End of VirtualCache demo.");
            }
        }
        const int MaxCacheEntries = 1000000;
        private void Populate(ObjectCache target)
        {
            _logger.Information("{0}: 
            Start of Populating target cache {1}.", DateTime.Now, target.Name);
            for (int i = 0; i < MaxCacheEntries; i++)
            {
                target.Set(string.Format("Hello{0}", i), string.Format("Value{0}", i), null);
            }
            _logger.Information("{0}: 
            End Populating target cache {1}.", DateTime.Now, target.Name);
        }
        private void RetrieveAll(ObjectCache source)
        {
            _logger.Information("{0}: 
            Start of RetrieveAll cache entries from {1}.", DateTime.Now, source.Name);
            for (int i = 0; i < MaxCacheEntries; i++)
            {
                if ((string)source[string.Format("Hello{0}", i)] == null)
                    Console.WriteLine("Failed, entry {0} was not found in cache {1}!", 
            string.Format("Hello{0}", source.Name));
            }
            _logger.Information("{0}: 
            End of RetrieveAll cache entries from {1}.", DateTime.Now, source.Name);
        }
    }
}

Instructions to create & build the sample program:

• Download & extract the sources from SOP v4.7 zip file.
• Open & build in release mode the SOP.sln.
• On a new VS solution, create a new C# console project.
• Add system reference to System.Runtime.Caching & DLL references to the newly built SOP.dll & SOP.Server.dll from the step above.
• Paste the above code snippet to the generated code file.
• Build and run the application.

Multi-Threaded

Virtual Cache is multi-thread safe. It has built-in locking mechanism that allows your code to use a Virtual Cache instance across different threads without worrying for data corruption.

Sample code that uses Virtual Cache in a multi-thread scenario:

/// <summary>
/// Heavily threaded Virtual Cache(VC) demo module.
/// </summary>
public class VirtualCacheThreaded
{
    private Log.Logger _logger;
    const int ThreadCount = 20;
    public void Run()
    {
        List<Action> actions = new List<Action>();
        List<VirtualCache> vcs = new List<VirtualCache>();
        // create threads that will populate Virtual Cache and retrieve the items.
        for (int i = 0; i < ThreadCount; i++)
        {
            // create a new VirtualCache instance where cached data is stored together
    // with the cached data key (clustered).
    var vc = new VirtualCache(string.Format("MyCacheStore{0}", i), true);
            if (_logger == null)
            {
                _logger = vc.Logger;
                _logger.LogLevel = Log.LogLevels.Verbose;
                _logger.Information("Start of VirtualCache demo.");
            }
            // function to execute by the thread.
            actions.Add(() =>
            {
                Populate(vc, i % 2 == 0);
                RetrieveAll(vc);
                var name = vc.Name;
                vc.Dispose();
                Console.WriteLine("VirtualCache {0} was disposed.", name);
            });
            vcs.Add(vc);
        }
        List<Task> tasks = new List<Task>();
        // launch or start the threads all at once.
        foreach (var a in actions)
        {
            var t = TaskRun(a);
            if (t == null)
                continue;
            tasks.Add(t);
        }
        // wait until all threads are finished.
        if (_threaded)
            Task.WaitAll(tasks.ToArray());
        Console.WriteLine("End of VirtualCache demo.");
    }
    private bool _threaded = true;
    private Task TaskRun(Action action)
    {
        if (!_threaded)
        {
            action();
            return null;
        }
        return Task.Run(action);
    }
    const int MaxCacheEntries = 50000;
    private void Populate(ObjectCache target, bool slidingExpiration = false)
    {
        _logger.Information("{0}: 
        Start of Populating target cache {1}.", DateTime.Now, target.Name);
        for (int i = 0; i < MaxCacheEntries; i++)
        {
            target.Set(string.Format("Hello{0}", i), string.Format("Value{0}", i), 
        new CacheItemPolicy() { SlidingExpiration = new TimeSpan(0, 15, 0) });
        }
        _logger.Information("{0}: 
        End Populating target cache {1}.", DateTime.Now, target.Name);
    }
    private void RetrieveAll(ObjectCache source)
    {
        _logger.Information("{0}: Start of RetrieveAll cache entries from {1}.", 
    DateTime.Now, source.Name);
        for (int i = 0; i < MaxCacheEntries; i++)
        {
            if ((string)source[string.Format("Hello{0}", i)] == null)
                Console.WriteLine("Entry {0} 
                was not found in cache {1}, 'it could have expired and got evicted.", 
        string.Format("Hello{0}", i), source.Name);
        }
        _logger.Information("{0}: End of RetrieveAll cache entries from {1}.", 
    DateTime.Now, source.Name);
    }
}

Data Persistence

Virtual Cache supports two modes of operation:

1. Memory Extender - In this mode, cached data are not persisted, the data files are used only to swap in/out of memory pageful of data.
2. Persistent Caching - In this mode, cached data are persisted & Virtual Cache makes them available even when the application and/or the computer is restarted.

Setting the 3^rd parameter of VirtualCache to true sets it to do persisted caching. It defaults to false.

Points of Interest

.NET 4.5 ObjectCache & VirtualCache API are good APIs to learn and use in our Apps development especially in authoring data caching solutions.

History

• 10/05/2015 - Changed to MIT license; attached sop.zip downloadable file.
• 10/09/2014 - Initial version. More discussions to follow, 'just wanted to get the message out about Virtual Cache availability