Distributed Caching using Redis Server with .NET/C# Client

• Download source code (Redis with Funq IoC on MVC 4)

Introduction

In this article, I would like to describe my experience with installing and configuring Redis server in most compact way. Also, I would like to do a brief overview of usage Redis hashes and lists in .NET/C# client.

In this article:

• Install Redis server (complete application files set is attached)
• Protecting the server (configure authentication)
• Configuring server replication
• Access cache from C# application
• Using ASP.NET session state with redis
• Sample usage of the Redis Sets, Lists and transactions
• Description of the attached sources (Redis with Funq IoC in MVC project : example)
• Caching optimization ideas

Background

Redis is one of the fastest and most feature-rich in-memory key value data stores.

Disadvantages

• No local data buffering (like synchronized local data buffering on the Azure caching)
• No full clusterization support yet (expected in the end of this year)

Advantages

• Easy configuration
• Simple usage
• High performance
• Support of different data types (like hashes, lists, sets, sorted sets)
• ASP.NET session integration
• Web UI for viewing content of the cache

In this simple demo, I'm going to demonstrate how to install and configure Redis on the server and use it from the C# code.

Redis Installation

Download binaries from https://github.com/dmajkic/redis/downloads (win32 win64 direct link) unpack archive to the application directory (e.g. C:\Program Files\Redis)

Download Redis service compiled from https://github.com/kcherenkov/redis-windows-service/downloads, then copy to the program folder (e.g. C:\Program Files\Redis. If config file is missing, download it and copy to the application directory as well. Example of the valid Redis config file is at https://raw.github.com/antirez/redis/2.6/redis.conf.

The complete set of files for Redis application is also available in zip file (x64).

When you have the full set of the application files (like it is shown on the image below),

navigate to the application directory and run the following command:

sc create %name% binpath= "\"%binpath%\" %configpath%" start= "auto" DisplayName= "Redis" 

Where:

• %name% -- name of service instance, example: redis-instance;
• %binpath% -- path to this project EXE file, example: C:\Program Files\Redis\RedisService_1.1.exe;
• %configpath% -- path to redis configuration file, example: C:\Program Files\Redis\redis.conf;

Example:

sc create Redis start= auto DisplayName= Redis binpath= "\"C:\Program Files\Redis\RedisService_1.1.exe\
" \"C:\Program Files\Redis\redis.conf\"" 

It should look like this:

Make sure that you have enough privileges to start the service. After installation, check that the service was created successfully and is running now:

Alternatively, you can use installer, created by someone (I've not tried): https://github.com/rgl/redis/downloads.

Redis Server Protection: Password, IP Filtering

The primary way to protect Redis server is to set IP filtering using Windows firewall or properties of the active network connection. Additional protection can be set using redis password. It needs to update the Redis config file (redis.conf) in the following way:

First, find the line:

# requirepass foobared   

Remove the # symbol in the beginning and replace foobared with new password:

requirepass foobared

Then restart Redis Windows service!!!

When instantiating the client, use constructor with a password:

RedisClient client = new RedisClient(serverHost, port, redisPassword);

Redis Server Replication (master - slave configuration)

This technique allows creation copy of the server data into the synchronized copy, this means that each time when master is modified, slave server gets notification and is automatically synchronized. Mostly replication is used for read (but not write) scalability or data redundancy and for the server failover. Setup two instances of Redis (two services on the same or different servers), then configure one of them as slave. To make Redis server instance to be slave of another server, change the config file in this way:

Find the line below:

# slaveof <masterip> <masterport>

replace with:

slaveof 192.168.1.1 6379

(specify real IP of the master server, and port in case you customized it). If master is configured to require password (authentication), change redis.conf as it is shown below, find line:

# masterauth <master-password>

remove the # symbol in the beginning and replace <master-password> with master password, to be like that:

masterauth mastpassword

Now this Redis instance can be used as a readonly synchronized copy of the master server.

Using Redis Cache from the C# Code

To use Redis in C# run the Manage NuGet packages addon, find ServiceStack.Redis pack, and install it.

Sample of using Set/Get methods directly from the instantiated client:

string host = "localhost";
string elementKey = "testKeyRedis";

using (RedisClient redisClient = new RedisClient(host))
{
      if (redisClient.Get<string>(elementKey) == null)
      {
           // adding delay to see the difference
           Thread.Sleep(5000); 
           // save value in cache
           redisClient.Set(elementKey, "some cached value");
      }
      // get value from the cache by key
      message = "Item value is: " + redisClient.Get<string>("some cached value");
 }

Typed entity sets are more interesting and practical, because they operate with exact types of objects. In the code sample below, there are two classes defined Phone, and Person - owner of the phone. Each phone instance has a reference to the owner. This code demonstrates how we can add, remove or find items in the cache by criteria:

public class Phone
{
   public int Id { get; set; }
   public string Model { get; set; }
   public string Manufacturer { get; set; }
   public Person Owner { get; set; }
}

public class Person
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Surname { get; set; }
    public int Age { get; set; }
    public string Profession { get; set; }
}

using (RedisClient redisClient = new RedisClient(host))
{
     IRedisTypedClient<phone> phones = redisClient.As<phone>();
     Phone phoneFive = phones.GetValue("5");
     if (phoneFive == null)
     {
          // make a small delay
          Thread.Sleep(5000);
          // creating a new Phone entry
          phoneFive = new Phone
          {
               Id = 5,
               Manufacturer = "Motorolla",
               Model = "xxxxx",
               Owner = new Person
               {
                    Id = 1,
                    Age = 90,
                    Name = "OldOne",
                    Profession = "sportsmen",
                    Surname = "OldManSurname"
               }
          };
          // adding Entry to the typed entity set
          phones.SetEntry(phoneFive.Id.ToString(), phoneFive);
     }
     message = "Phone model is " + phoneFive.Manufacturer;
     message += "Phone Owner Name is: " + phoneFive.Owner.Name;
}

In the example above, we instantiate the typed client IRedisTypedClient<phone>, which works with specific type of cached objects: Phone type.

ASP.NET Session State with Redis

To configure ASP.NET session state with redis provider, add a new file to your web project, named RedisSessionStateProvider.cs, copy code from https://github.com/chadman/redis-service-provider/raw/master/RedisProvider/SessionProvider/RedisSessionProvider.cs, then add or change the following section in the configuration file (sessionState tag has to be inside system.web tag), or you can download attached sources and copy code.

<sessionstate timeout="1" mode="Custom" 
customprovider="RedisSessionStateProvider" cookieless="false">
      <providers>
        <add name="RedisSessionStateProvider" writeexceptionstoeventlog="false" 
        type="RedisProvider.SessionProvider.CustomServiceProvider" 
        server="localhost" port="6379" password="pasword">
      </add> </providers>
</sessionstate>

NOTE, that password is optional, based on the server authentication. It must be replaced with real value, or removed, if Redis server doesn't require authentication. server attribute and port also have to be replaced according to concrete values (default port is 6379). Then in the project, you can use the session state:

// in the Global.asax
public class MvcApplication1 : System.Web.HttpApplication
{
    protected void Application_Start()
    {
        //....
    }

    protected void Session_Start()
    {
        Session["testRedisSession"] = "Message from the redis ression";
    }
}

In the Home controller:

public class HomeController : Controller
{
    public ActionResult Index()
    {
       //...
       ViewBag.Message = Session["testRedisSession"];
       return View();
    }
//...
}

Result:

ASP.NET output cache provider with redis can be configured in the similar way.

Redis Sets and Lists

The major note is that Redis lists implement IList<T> while Redis sets implement ICollection<T>. Let's see how we can use them.

Lists are mostly used when it needs to separate different categories of objects of the same type. For example, we have "most selling phones" and "old collection" two lists of phones:

string host = "localhost";
using (var redisClient = new RedisClient(host))
{
    //Create a 'strongly-typed' API that makes all Redis Value operations to apply against Phones
    IRedisTypedClient<phone> redis = redisClient.As<phone>();

    IRedisList<phone> mostSelling = redis.Lists["urn:phones:mostselling"];
    IRedisList<phone> oldCollection = redis.Lists["urn:phones:oldcollection"];

    Person phonesOwner = new Person
        {
            Id = 7,
            Age = 90,
            Name = "OldOne",
            Profession = "sportsmen",
            Surname = "OldManSurname"
        };
                
    // adding new items to the list
    mostSelling.Add(new Phone
            {
                Id = 5,
                Manufacturer = "Sony",
                Model = "768564564566",
                Owner = phonesOwner
            });

    oldCollection.Add(new Phone
            {
                Id = 8,
                Manufacturer = "Motorolla",
                Model = "324557546754",
                Owner = phonesOwner
            });

    var upgradedPhone  = new Phone
    {
        Id = 3,
        Manufacturer = "LG",
        Model = "634563456",
        Owner = phonesOwner
    };

    mostSelling.Add(upgradedPhone);

    // remove item from the list
    oldCollection.Remove(upgradedPhone);

    // find objects in the cache
    IEnumerable<phone> LGPhones = mostSelling.Where(ph => ph.Manufacturer == "LG");

    // find specific
    Phone singleElement = mostSelling.FirstOrDefault(ph => ph.Id == 8);

    //reset sequence and delete all lists
    redis.SetSequence(0);
    redisClient.Remove("urn:phones:mostselling");
    redisClient.Remove("urn:phones:oldcollection");
}

Redis sets are useful when it needs to store associated sets of data and gather statistical information, for example answer -> queustion, votes for an answer or question. Let's say that we have questions and answers, it needs to store them in the cache for better performance. Using Redis, we can do it this way:

/// <summary>
/// Gets or sets the Redis Manager. The built-in IoC used with ServiceStack autowires this property.
/// </summary>
IRedisClientsManager RedisManager { get; set; }
/// <summary>
/// Delete question by performing compensating actions to 
/// StoreQuestion() to keep the datastore in a consistent state
/// </summary>
/// <param name="questionId">
public void DeleteQuestion(long questionId)
{
    using (var redis = RedisManager.GetClient())
    {
        var redisQuestions = redis.As<question>();

        var question = redisQuestions.GetById(questionId);
        if (question == null) return;
                
        //decrement score in tags list
        question.Tags.ForEach(tag => redis.IncrementItemInSortedSet("urn:tags", tag, -1));

        //remove all related answers
        redisQuestions.DeleteRelatedEntities<answer>(questionId);

        //remove this question from user index
        redis.RemoveItemFromSet("urn:user>q:" + question.UserId, questionId.ToString());

        //remove tag => questions index for each tag
        question.Tags.ForEach("urn:tags>q:" + tag.ToLower(), questionId.ToString()));

        redisQuestions.DeleteById(questionId);
    }
}

public void StoreQuestion(Question question)
{
    using (var redis = RedisManager.GetClient())
    {
        var redisQuestions = redis.As<question>();

        if (question.Tags == null) question.Tags = new List<string>();
        if (question.Id == default(long))
        {
            question.Id = redisQuestions.GetNextSequence();
            question.CreatedDate = DateTime.UtcNow;

            //Increment the popularity for each new question tag
            question.Tags.ForEach(tag => redis.IncrementItemInSortedSet("urn:tags", tag, 1));
        }

        redisQuestions.Store(question);
        redisQuestions.AddToRecentsList(question);
        redis.AddItemToSet("urn:user>q:" + question.UserId, question.Id.ToString());

        //Usage of tags - Populate tag => questions index for each tag
        question.Tags.ForEach(tag => redis.AddItemToSet
        ("urn:tags>q:" + tag.ToLower(), question.Id.ToString()));
    }
}

/// <summary>
/// Delete Answer by performing compensating actions to 
/// StoreAnswer() to keep the datastore in a consistent state
/// </summary>
/// <param name="questionId">
/// <param name="answerId">
public void DeleteAnswer(long questionId, long answerId)
{
    using (var redis = RedisManager.GetClient())
    {
        var answer = redis.As<question>().GetRelatedEntities<answer>
        (questionId).FirstOrDefault(x => x.Id == answerId);
        if (answer == null) return;
                
        redis.As<question>().DeleteRelatedEntity<answer>(questionId, answerId);
                
        //remove user => answer index
        redis.RemoveItemFromSet("urn:user>a:" + answer.UserId, answerId.ToString());
    }
}

public void StoreAnswer(Answer answer)
{
    using (var redis = RedisManager.GetClient())
    {
        if (answer.Id == default(long))
        {
            answer.Id = redis.As<answer>().GetNextSequence();
            answer.CreatedDate = DateTime.UtcNow;
        }

        //Store as a 'Related Answer' to the parent Question
        redis.As<question>().StoreRelatedEntities(answer.QuestionId, answer);
        //Populate user => answer index
        redis.AddItemToSet("urn:user>a:" + answer.UserId, answer.Id.ToString());
    }
}

public List<answer> GetAnswersForQuestion(long questionId)
{
    using (var redis = RedisManager.GetClient())
    {
        return redis.As<question>().GetRelatedEntities<answer>(questionId);
    }
}

public void VoteQuestionUp(long userId, long questionId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register upvote against question and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:q>user+:" + questionId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:q>user-:" + questionId, userId.ToString()));

        //Register upvote against user and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:user>q+:" + userId, questionId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>q-:" + userId, questionId.ToString()));
    });
}

public void VoteQuestionDown(long userId, long questionId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register downvote against question and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:q>user-:" + questionId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:q>user+:" + questionId, userId.ToString()));

        //Register downvote against user and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet"urn:user>q-:" + userId, questionId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>q+:" + userId, questionId.ToString()));
    });
}

public void VoteAnswerUp(long userId, long answerId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register upvote against answer and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:a>user+:" + answerId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:a>user-:" + answerId, userId.ToString()));

        //Register upvote against user and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:user>a+:" + userId, answerId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>a-:" + userId, answerId.ToString()));
    });
}

public void VoteAnswerDown(long userId, long answerId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register downvote against answer and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:a>user-:" + answerId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:a>user+:" + answerId, userId.ToString()));

        //Register downvote against user and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:user>a-:" + userId, answerId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>a+:" + userId, answerId.ToString()));
    });
}

public QuestionResult GetQuestion(long questionId)
{
    var question = RedisManager.ExecAs<question>
    (redisQuestions => redisQuestions.GetById(questionId));
    if (question == null) return null;

    var result = ToQuestionResults(new[] { question })[0];
    var answers = GetAnswersForQuestion(questionId);
    var uniqueUserIds = answers.ConvertAll(x => x.UserId).ToHashSet();
    var usersMap = GetUsersByIds(uniqueUserIds).ToDictionary(x => x.Id);

    result.Answers = answers.ConvertAll(answer =>
        new AnswerResult { Answer = answer, User = usersMap[answer.UserId] });

    return result;
}

public List<user> GetUsersByIds(IEnumerable<long> userIds)
{
    return RedisManager.ExecAs<user>(redisUsers => redisUsers.GetByIds(userIds)).ToList();
}

public QuestionStat GetQuestionStats(long questionId)
{
    using (var redis = RedisManager.GetReadOnlyClient())
    {
        var result = new QuestionStat
        {
            VotesUpCount = redis.GetSetCount("urn:q>user+:" +questionId),
            VotesDownCount = redis.GetSetCount("urn:q>user-:" + questionId)
        };
        result.VotesTotal = result.VotesUpCount - result.VotesDownCount;
        return result;
    }
}

public List<tag> GetTagsByPopularity(int skip, int take)
{
    using (var redis = RedisManager.GetReadOnlyClient())
    {
        var tagEntries = redis.GetRangeWithScoresFromSortedSetDesc("urn:tags", skip, take);
        var tags = tagEntries.ConvertAll(kvp => new Tag { Name = kvp.Key, Score = (int)kvp.Value });
        return tags;
    }
}

public SiteStats GetSiteStats()
{
    using (var redis = RedisManager.GetClient())
    {
        return new SiteStats
        {
            QuestionsCount = redis.As<question>().TypeIdsSet.Count,
            AnswersCount = redis.As<answer>().TypeIdsSet.Count,
            TopTags = GetTagsByPopularity(0, 10)
        };
    }
}

Attached Sources Description

List of included packages is in the packages.config,
Funq IoC configuration, registering types and current controller factory - are in the Global.asax (property dependency injection)
Usage of a simple client - in the home controller
Usage of IoC based cache in the Question and Answer controllers, and Global.asax application file. To see how it works, you can run the project, and open in the browser following URL: http://localhost:37447/Question/GetQuestions?tag=test .
You can play with tags, like test3, test11, test2, etc.
Redis Cache configuration - in the web config (<system.web><sessionState> section) and in the RedisSessionStateProvider.cs file.
There are a lot of TODOs in the MVC project, so if you want to improve/continue please update it, and upload.

I would much appreciate if someone could help build MVC application with simple UI, using Redis (with Funq IoC) cache. Funq IoC is already configured, example of the usage is in the Question controller.

NOTE: Samples were partially taken from the "ServiceStack.Examples-master" solution

Conclusion. Optimization Caching in Application with Fast Local Cache

Since Redis doesn't store data locally (no local replication), it might make sence to optimize performance by storing some light or user - dependent objects in the local cache (to skip serialization to string and client - server data transfering). For example, in the web application, it is better to use 'System.Runtime.Caching.ObjectCache' for light objects, which are user dependent and used frequently by the application. Otherwise, when object has common usage, large size it must be saved in the distributed Redis cache. Example of the user dependent objects - profile information, personalization information. Common objects - localization data, information shared between different users, etc.

• Download source code (Redis with Funq IoC on MVC 4)

Links

• How to run redis service description:
  https://github.com/kcherenkov/redis-windows-service
• Documentation:
  http://redis.io/documentation
• .NET/C# examples:
  https://github.com/ServiceStack/ServiceStack.Examples
• Suggested nice articles to get started with Redis on Windows using C#:
  http://maxivak.com/getting-started-with-redis-and-asp-net-mvc-under-windows/:
  http://www.piotrwalat.net/using-redis-with-asp-net-web-api/
• Redis about:
  https://github.com/ServiceStack/ServiceStack.Redis
• Azure caching
  http://kotugoroshko.blogspot.ae/2013/07/windows-azure-caching-integration.html