Provide high-level abstractions for common Redis key types over the StackExchange.Redis low-level API.
Introduction
There are already several Redis client libraries for .NET - StackExchange.Redis, Microsoft.Extensions.Caching.Redis and ServiceStack.Redis to name the most popular - so why write another one? I wanted a few things in a Redis client library:
- a "model" of my application's cache, similar to the
DbContext in EF - automatic handling of POCO data types, plus easy support for other data primitives
- help with consistent key naming
- support for a key "
namespace" - easy identification of a key's type and content
- Intellisense to show only the commands allowed for the key type
These objectives led to the design of a context/container called RedisContainer, holding strongly-typed data objects modeling Redis key types. The RedisContainer provides a key namespace and allows for an intuitive model of the Redis keys used within an application, and optionally keeps track of the keys used but does not itself cache any data. The strongly-typed objects also do not cache any data in application memory, but instead encapsulate only the commands specific to each of the common data types:
| Class | Redis data type |
RedisItem<T> | binary safe string |
RedisBitmap | bit array |
RedisList<T> | list |
RedisSet<T> | set |
RedisSortedSet<T> | zset |
RedisHash<K, V> | hash |
RedisDtoHash<T> | map a hash as a DTO |
RedisObject | *base class for all key types |
The library is dependent on StackExchange.Redis for all communication to the Redis server, and the API supports asynchronous I/O only.
Usage
Basics
Create a connection and container. The RedisConnection takes a StackExchange configuration string. The RedisContainer takes a connection and an optional namespace for all keys.
var cn = new RedisConnection("127.0.0.1:6379,abortConnect=false");
var container = new RedisContainer(cn, "test");
Keys are managed by the container. The key may already exist in the Redis database. Or not. The GetKey method does not call Redis. If the container is tracking key creation and the key was already added to the container, then that object is returned, otherwise a new RedisObject of the type requested is created and returned.
var key1 = container.GetKey<RedisItem<string>>("key1");
var key2 = container.GetKey<RedisItem<int>>("key2");
The generic parameter for any type can be an IConvertible, byte[] or POCO/DTO. Example:
var longitem = container.GetKey<RedisItem<long>>("longitem");
var intlist = container.GetKey<RedisList<int>("intlist");
var customers = container.GetKey<RedisHash<string, Customer>>("customers");
var cust1 = container.GetKey<RedisDtoHash<Customer>>("cust1");
Automatic JSON serialization/deserialization of POCO types:
var key3 = container.GetKey<RedisItem<Customer>>("key3");
await key3.Set(new Customer { Id = 1, Name = "freddie" });
var aCust = await key3.Get();
All key types support basic commands:
key1.DeleteKey()
key1.Expire(30)
key1.ExpireAt(DateTime.Now.AddHours(1))
key1.IdleTime()
key1.KeyExists()
key1.Persist()
key1.TimeToLive()
Access to the StackExchange.Redis.Database is available to directly execute any commands not supported by the RedisProvider. Example:
var randomKey = container.Database.KeyRandom();
Templated Key Creation
When using the common pattern of including the object Id in the key name, for example "user:1" or "user:1234", manually creating each key and ensuring both the data type and key name format are correct can be error prone. The KeyTemplate<T> acts as a factory for keys of the specified type and key name pattern.
var docCreator = container.GetKeyTemplate<RedisItem<string>>("doc:{0}");
var doc1 = docCreator.GetKey(1);
var doc2 = docCreator.GetKey(2);
Transactions and Batches
Pipelines are supported via StackExchange.Redis-based transactions and batches. Use the RedisContainer to create a batch or transaction, and then add queued tasks using WithBatch() or WithTransaction().
var key1 = container.GetKey<RedisSet<string>>("key1");
var batch = container.CreateBatch();
key1.WithBatch(batch).Add("a");
key1.WithBatch(batch).Add("b");
await batch.Execute();
var keyA = container.GetKey<RedisItem<string>>("keya");
var keyB = container.GetKey<RedisItem<string>>("keyb");
await keyA.Set("abc");
await keyB.Set("def");
var tx = container.CreateTransaction();
var task1 = keyA.WithTx(tx).Get();
var task2 = keyB.WithTx(tx).Get();
await tx.Execute();
var a = task1.Result;
var b = task2.Result;
Alternately, you can add a task directly to the transaction or batch with the syntax shown below:
var keyA = container.GetKey<RedisItem<string>>("keya");
var keyB = container.GetKey<RedisItem<string>>("keyb");
await keyA.Set("abc");
await keyB.Set("def");
var tx = container.CreateTransaction();
tx.AddTask(() => keyA.Get());
tx.AddTask(() => keyB.Get());
await tx.Execute();
var task1 = tx.Tasks[0] as Task<string>;
var task2 = tx.Tasks[1] as Task<string>;
var a = task1.Result;
var b = task2.Result;
Strongly-Typed Data Objects
RedisItem<T> and RedisBitmap
The Redis binary safe string. RedisBitmap is a RedisItem<byte[]> adding bit manipulation operations. RedisValueItem is a RedisItem<RedisValue> which can be used when the generic parameter type is unimportant.
| RedisItem<T> | Redis command |
| Get and set: | |
Get(T) | GET |
Set(T, [TimeSpan], [When]) | SET, SETEX, SETNX |
GetSet(T) | GETSET |
GetRange(long, long) | GETRANGE |
SetRange(long, T) | SETRANGE |
GetMultiple(IList<RedisItem<T>>) | MGET |
SetMultiple(IList<KeyValuePair<RedisItem<T>, T>> | MSET, MSETNX |
| String-related: | |
Append(T) | APPEND |
StringLength() | STRLEN |
| Numeric-related: | |
Increment([long]) | INCR, INCRBY |
Decrement([long]) | DECR, DECRBY |
| RedisBitmap: | |
GetBit(long) | GETBIT |
SetBit(long, bool) | SETBIT |
BitCount([long], [long]) | BITCOUNT |
BitPosition(bool, [long], [long]) | BITPOS |
BitwiseOp(Op, RedisBitmap, ICollection<RedisBitmap>) | BITOP |
RedisList<T>
A LIST in Redis is a collection of elements sorted according to the order of insertion. Use RedisValueList when list items are not of the same type.
| RedisList<T> | Redis command |
| Add and remove: | |
AddBefore(T, T) | LINSERT BEFORE |
AddAfter(T, T) | LINSERT AFTER |
AddFirst(params T[]) | LPUSH |
AddLast(params T[]) | RPUSH |
Remove(T, [long]) | LREM |
RemoveFirst() | LPOP |
RemoveLast() | RPOP |
| Indexed access: | |
First() | LINDEX 0 |
Last() | LINDEX -1 |
Index(long) | LINDEX |
Set(long, T) | LSET |
Range(long, long) | LRANGE |
Trim(long, long) | LTRIM |
| Miscellaneous: | |
Count() | LLEN |
PopPush(RedisList<T>) | RPOPLPUSH |
Sort | SORT |
SortAndStore | SORT .. STORE |
GetAsyncEnumerator() | |
RedisSet<T>
A SET in Redis is a collection of unique, unsorted elements. Use RedisValueSet when set items are not of the same type.
| RedisSet<T> | Redis command |
| Add and remove: | |
Add(T) | SADD |
AddRange(IEnumerable<T>) | SADD |
Remove(T) | SREM |
RemoveRange(IEnumerable<T>) | SREM |
Pop([long]) | SPOP |
Peek([long]) | SRANDMEMBER |
Contains(T) | SISMEMBER |
Count() | SCARD |
| Set operations: | |
Sort | SORT |
SortAndStore | SORT .. STORE |
Difference | SDIFF |
DifferenceStore | SDIFFSTORE |
Intersect | SINTER |
IntersectStore | SINTERSTORE |
Union | SUNION |
UnionStore | SUNIONSTORE |
| Miscellaneous: | |
ToList() | SMEMBERS |
GetAsyncEnumerator() | SSCAN |
RedisSortedSet<T>
The ZSET in Redis is similar to the SET, but every element has an associated floating number value, called score. Use RedisSortedValueSet when set items are not of the same type.
| RedisSortedSet<T> | Redis command |
| Add and remove: | |
Add(T, double) | ZADD |
AddRange(IEnumerable<(T, double)>) | ZADD |
Remove(T) | ZREM |
RemoveRange(IEnumerable<(T, double)>) | ZREM |
RemoveRangeByScore | ZREMRANGEBYSCORE |
RemoveRangeByValue | ZREMRANGEBYLEX |
RemoveRange([long], [long]) | ZREMRANGEBYRANK |
| Range and count: | |
Range([long], [long], [Order]) | ZRANGE |
RangeWithScores([long], [long], [Order]) | ZRANGE ... WITHSCORES |
RangeByScore | ZRANGEBYSCORE |
RangeByValue | ZRANGEBYLEX |
Count() | ZCARD |
CountByScore | ZCOUNT |
CountByValue | ZLEXCOUNT |
| Miscellaneous: | |
Rank(T, [Order]) | ZRANK, ZREVRANK |
Score(T) | ZSCORE |
IncrementScore(T, double) | ZINCRBY |
Pop([Order]) | ZPOPMIN, ZPOPMAX |
| Set operations: | |
Sort | SORT |
SortAndStore | SORT .. STORE |
IntersectStore | ZINTERSTORE |
UnionStore | ZUNIONSTORE |
GetAsyncEnumerator() | ZSCAN |
RedisHash<TKey, TValue>
The Redis HASH is a map composed of fields associated with values. The RedisHash<TKey, TValue> treats the hash as a dictionary of strongly typed key-value pairs. The RedisValueHash can be used to store different data types in keys and values, while the RedisDtoHash<TDto> maps the properties of a DTO to the fields of a hash.
| RedisHash<TKey,TValue> | Redis command |
| Get, set and remove: | |
Get(TKey) | HGET |
GetRange(ICollection<TKey>) | HMGET |
Set(TKey, TValue, [When]) | HSET, HSETNX |
SetRange(ICollection<KeyValuePair<TKey, TValue>>) | HMSET |
Remove(TKey) | HDEL |
RemoveRange(ICollection<TKey>) | HDEL |
| Hash operations: | |
ContainsKey(TKey) | HEXISTS |
Keys() | HKEYS |
Values() | HVALS |
Count() | HLEN |
Increment(TKey, [long]) | HINCRBY |
Decrement(TKey, [long]) | HINCRBY |
| Miscellaneous: | |
ToList() | HGETALL |
GetAsyncEnumerator() | HSCAN |
| RedisDtoHash<TDto> | |
FromDto<TDto> | HSET |
ToDto() | HMGET |
Sample Application
Redis documentation provides a tutorial of a simple Twitter clone and an e-book with a more developed app. The sample is based on the Redis concepts described in these.
The sample, "Twit", is a very basic Blazor webassembly application. The part we're interested in here is the CacheService, which uses the RedisProvider to model and manage the Redis cache.
public class CacheService
{
private readonly RedisContainer _container;
private RedisItem<long> NextUserId;
private RedisItem<long> NextPostId;
private RedisHash<string, long> Users;
private RedisHash<string, long> Auths;
private RedisList<Post> Timeline;
private KeyTemplate<RedisDtoHash<User>> UserTemplate;
private KeyTemplate<RedisDtoHash<Post>> PostTemplate;
private KeyTemplate<RedisSortedSet<long>> UserProfileTemplate;
private KeyTemplate<RedisSortedSet<long>> UserFollowersTemplate;
private KeyTemplate<RedisSortedSet<long>> UserFollowingTemplate;
private KeyTemplate<RedisSortedSet<long>> UserHomeTLTemplate;
...
}
The CacheService here contains the RedisContainer, but it could just as easily extend the RedisContainer instead: public class CacheService : RedisContainer {}
In either case, the container is provided with the connection information and keyNamespace, which in this case is "twit". All key names created by the container will be in the format "twit:{keyname}".
Here, we see what I call "fixed" keys, keys whose names are constant, and "dynamic" keys, those whose names include an Id or other variable data.
So NextUserId and NextPostId are simple "binary safe string" items, which we know contain a long integer. These fields are used to obtain the Ids for newly-created users and posts:
NextUserId = _container.GetKey<RedisItem<long>>("nextUserId");
NextPostId = _container.GetKey<RedisItem<long>>("nextPostId");
var userid = await NextUserId.Increment();
var postid = await NextPostId.Increment();
Users and Auths are hashes, used like simple dictionaries to map user name or authentication "ticket" strings to a user id.
Users = _container.GetKey<RedisHash<string, long>>("users");
Auths = _container.GetKey<RedisHash<string, long>>("auths");
await Users.Set(userName, userid);
var userid = Users.Get(userName);
Timeline is a list of the Post POCO type. (The sample includes several timelines, usually stored as sets. This list is more illustrative than useful.)
Timeline = _container.GetKey<RedisList<Post>>("timeline");
var data = new Post {
Id = id, Uid = userid, UserName = userName, Posted = DateTime.Now, Message = message };
await Timeline.AddFirst(data);
Now for the "dynamic" keys. We'll maintain a hash for each user and post, with key names containing the ids. The KeyTemplate<T> allows us to define the key type and the format of the key name once, and then retrieve individual keys as needed. The hash keys here also automatically map to POCO/DTO types, where the properties of the POCO are fields in the stored hash.
UserTemplate = _container.GetKeyTemplate<RedisDtoHash<User>>("user:{0}");
PostTemplate = _container.GetKeyTemplate<RedisDtoHash<Post>>("post:{0}");
var user = UserTemplate.GetKey(userId);
var post = PostTemplate.GetKey(postId);
var userData = new User {
Id = userId, UserName = name, Signup = DateTime.Now, Password = pwd, Ticket = ticket
};
user.FromDto(userData);
var postData = new Post {
Id = postId, Uid = userid, UserName = userName, Posted = DateTime.Now, Message = message
};
post.FromDto(postData);
Finally, the model contains templates for several sorted sets (ZSETs) which are keyed by the user Id.
UserProfileTemplate = _container.GetKeyTemplate<RedisSortedSet<long>>("profile:{0}");
UserFollowersTemplate = _container.GetKeyTemplate<RedisSortedSet<long>>("followers:{0}");
UserFollowingTemplate = _container.GetKeyTemplate<RedisSortedSet<long>>("following:{0}");
UserHomeTLTemplate = _container.GetKeyTemplate<RedisSortedSet<long>>("home:{0}");
With these in place, a user with Id=1 would have the following keys:
RedisDtoHash<User>("user:1")
RedisSortedSet<long>("profile:1")
RedisSortedSet<long>("home:1")
RedisSortedSet<long>("following:1")
RedisSortedSet<long>("followers:1")
So, a simple model here, and easy to conceptualize because of the strongly-typed key fields and templates. It's a good time to note that the RedisContainer will keep track of these keys, but if there are many - for example, thousands of users and posts - you probably don't want to have the container maintain a dictionary of all these keys.
The CacheService provides RegisterUser, LoginUser, CreatePost, GetTimeline and FollowUser functionality similar to that in the e-book mentioned above, and I'll leave them for anyone interested to explore on their own. Here's one last snippet showing the RegisterUser logic:
public async Task<string> RegisterUser(string name, string pwd)
{
if ((await Users.ContainsKey(name))) throw new Exception("User name already exists");
var id = await NextUserId.Increment();
var user = UserTemplate.GetKey(id);
var ticket = Guid.NewGuid().ToString();
var userData = new User {
Id = id, UserName = name, Signup = DateTime.Now, Password = pwd, Ticket = ticket };
var tx = _container.CreateTransaction();
user.WithTx(tx).FromDto(userData);
Users.WithTx(tx).Set(name, id);
Auths.WithTx(tx).Set(ticket, id);
await tx.Execute();
return ticket;
}
Points of Interest
Why async only? Because I/O operations should be async, and while Redis (and StackExchange.Redis) are super fast, it's always good to remember that Redis is not a local in-process cache.
Why doesn't the API use "*Async" named methods? Because I don't like them.
There are still some pain points in the RedisProvider but overall I'm finding it an improvement over the basic StackExchange API. The syntax for transactions (and batches) is clumsy, and just like StackExchange requires that you add async tasks but not await them, which often results in a lot of irritating CS4014 "Because this call is not awaited ..." compiler warnings. These can be disabled with a pragma, but can still make the code more error prone.
Other Redis data types or features - HyperLogLogs, GEO, streams and Pub/Sub - are not currently supported.
I'd originally planned to have these strongly-typed data objects implement .NET interfaces, IEnumerable<T> at the least, and IList<T>, ISet<T> and IDictionary<K, V> as appropriate to provide familiar .NET semantics. The first version of RedisProvider offered only a synchronous API and did implement .NET interfaces, but there were two main problems. First, I found frequent Redis to .NET "impedance mismatches" in what a Redis key type supported and a seemingly complementary interface demanded. Second, and more important to me given my objective of keeping the Intellisense scoped to only the commands available for the key type, was the huge amount of extension methods brought in by System.Linq when implementing IEnumerable<T> or any of its sub-interfaces. Given that these objects don't hold data locally, most of these methods would be very inefficient if called, and needlessly cluttered API discovery.
Github repository is here.
History
- 2nd June, 2020: Initial version
