In an earlier blog post I discussed the many benefits of a distributed architecture
of expert systems. This post is about implementing this architecture to solve a problem we all have. Consider the act of sending notifications.
It is another typical tasks that most development projects include in some way (the other one being <link> logging that I've written about earlier).
It can be used to notify users and administrators of state changes that occurred in an application particularly when exception are raised
and a variety of other reasons amongst them, automatically send business reports to vested parties or send auto generated confirmation emails to end users.
Notifications however also represent significant security risks. Imagine those automated reports being emailed to a business persons personal email address
and then that person joins the competition. Even with policies in place, it's difficult to ensure that nothing falls between the cracks. In addition,
those same application that log that vulnerable application data often send that same information to users in emails. Now anyone with the right equipment can pick those up
and use it to compromise those systems. To summarize, if we were going to build an expert notification system, it would have to protect us from these email violations
- Emails containing information about the internal working of applications can not be sent to recipients outside the corporate network.
- Emails containing confidential customer information such as social security numbers, credit card numbers etc can not be sent to people outside of the corporate network
- Emails containing confidential financial business reports. Business applications often include reports measuring the business units performance.
These reports are also often automatically sent out to managers. These also can not be sent to users outside of the corporate network.
These do not represent all of the security risks notifications can include, but indicate that these systems need to be tightly controlled and managed.
Additionally, almost every company I've ever consulted with has had a situation of an out of control application sending thousands or millions of exception
emails to users, bringing the corporate email exchange to it's knees. An expert system is needed, one that is dedicated to monitoring what is sent
out and making sure that the security of the company isn't being compromised.
Expert Systems and the FaME pattern
The approach to designing a system like that is to leverage messaging to design a very flexible solution that is both scalable, fault tolerant and decoupled.
A generic approach to implementing such a solution is to have applications implement the desired features by integrating with a Facade to the Expert System.
The Facade sends the information to a Message bus and the Expert system receives the information and processes it.
The advantages of this system is that the transmitting and receiver applications are completely decoupled from each other. The receiver can be maintained,
developed and scaled completely independently from the clients that use it. As long as the recipient can read the message sent by the transmitters,
the recipient can go through its' release cycles without affecting the deployed applications that make use of it in any way.
The FaME design approach to Notifications and Logging looks as follows.
The diagram above shows how various applications send notification and log messages to exchanges. The LogManager reads the queue, analyzes the message
and saves it in a secured centralized storage environment. The NotificationManager, analyzes the messages coming off its queue, those that violate the email
policy are rejected and notifications sent to the applications. Throttling rules are also applied to ensure that the infrastructure is not overwhelmed with processes
gone amok. If everything checks out, the notification is sent to the email exchange.
Technical Design
The following is the layer design of this system. The Notification and Log wrappers are bundled in a Utilities assembly.
The Gateway solution includes all of the technologies necessary to communicate
with the message bus and is incorporated into a Infrastructure assembly that will also include interface components with other infrastructure systems such as a Database.
The expert utility systems are wrapped up in a Monitor assembly that will include other standard utility type expert systems.
The gateway solution that's referred to above is documented in an earlier blog that you can read here.
Implementation
The Notification Facade is extremely simple to implement. All they do is take the information and serialize them to a message that the messagebus can handle.
Here's an example of the Notification Facade, using the Gateway implementation.
But first we define the notification interface, for the purposes of this example we define an extremely simple interface:
public interface INotify
{
void Transmit(Notification message);
}
The Interface is implemented by a Notifier class that implements the Transmit method as follows:
public void Transmit(Notification message)
{
server.Send(message);
}
And the serialization of the Notification message implemented by a JSON serializer as follows:
public Message ConvertObjectToMessage(RabbitMQ.Client.IModel channel, object packetToSend)
{
Notification message = (Notification)packetToSend;
if (channel == null)
throw new ApplicationException("The channel is null, " +
"a valid channel is required to create a message");
byte[] bytes = null;
var properties = channel.CreateBasicProperties();
var json = JsonConvert.SerializeObject(message);
bytes = Encoding.GetEncoding(CharSet).GetBytes(json);
properties.ContentType = CONTENT_TYPE;
properties.ContentEncoding = CharSet;
properties.Headers = new Dictionary<string, string>();
properties.Headers.Add("type", message.GetType().ToString());
return new Message() { Body = bytes, Properties = properties, RoutingKey = string.Empty };
}
The converter uses the JSON serializer (see line 9). All that remains is to set afew properties and the routing key appropriately (lines 11 - 13).
I'm not an expert on the trigger words that should block emails, that would not only be a project unto itself, it would probably constitute an entire business model.
However, the general solution would take the form below. A list of words will trigger warnings, those words I have in one regular expression.
If there are regex performance experts out there, I'd be curious to know if a long list of or'd expressions is more efficient than running through
a long list of regular expressions of one word. I'm assuming the OR'd way is the way to go. Here we have regex lists for the three violations mentioned earlier.
namespace YEA.Monitor.NotificationManager
{
public interface IContentAnalyzer
{
ContentFlag Analyze(string content);
}
public class ContentAnalyzer : IContentAnalyzer
{
private Regex BlockedWords;
private Regex IUOWords;
private Regex ApplicationData;
public ContentAnalyzer()
{
RegexOptions options = RegexOptions.IgnoreCase | RegexOptions.Singleline;
var blockedPattern = @"(credit\s+card|creditcard|\d{14})";
var iuoPattern = @"(report|social\s+security\s+number|ss#|\d{3}\s+\d{2}\s+\d{3})";
var appData = @"(stacktrace|exception)";
BlockedWords = new Regex(blockedPattern, options);
IUOWords = new Regex(iuoPattern, options);
ApplicationData = new Regex(appData, options);
}
public ContentFlag Analyze(string content)
{
ContentFlag result = ContentFlag.Ok;
if (BlockedWords.IsMatch(content))
result |= ContentFlag.Blocked;
if (IUOWords.IsMatch(content))
result |= ContentFlag.InternalUseOnly;
if (ApplicationData.IsMatch(content))
result |= ContentFlag.InternalUseOnly;
return result;
}
}
}
The result is an enumeration flag that can be combined the same way as RegexOptions. I found a nice blog that told me exactly how
to create an enumerator that behaves as a bit flag here.
Not that it's really needed, but it's cool and using it isn't entirely ridiculous.
[Flags]
public enum ContentFlag
{
Blocked,
InternalUseOnly,
Ok
}
The bit flag is used to determine the result of the analysis, and that is used to determine
whether to send the notification or block it and respond with an error.
public ContentFlag Check(Notification notification)
{
var content = notification.Subject + " " + notification.Body;
var result = Analyzer.Analyze(content);
bool isOutgoingMail = checkEmailAddresses(notification);
return isOutgoingMail ? result : ContentFlag.Ok;
}
Line 4 returns the result from a content analyzer, it determines if the content violates any of the requirements noted at the beginning of this blog.
That result is returned in this check only if one of the email recipients is outside of the network. We could also have pulled that email out of the recipient list
and return it some sort of hybrid result to the caller, letting it know that there was an outside recipient, but that it was pulled. In this case we've decided
not to do that, but to just return an error to the sender for the entire post. The notification (given the nature of the system) is probably automated and having
internal emails go to an outside recipient is never a good idea - good habits need to be established.
var result = Check(notification);
if (result > ContentFlag.Ok)
SendErrorNotification(ContentFlag.Blocked);
else
client.SendAsync(notification, deliveryTag);
Based on the result, the email is either sent (Line 5) or an error notification is sent instead.
Here is where you can find the enterprise solution for distributed expert systems discussed in this blog.
