Before We Start
It is important to note that we are implementing inbound messaging between the bot and our Azure Functions using storage queues for two reasons. Firstly, and most importantly, we are purely just shipping messages into our processing layer so we don’t need any additional features. Secondly, when a Discord bot becomes large enough, it will get sharded, which allows the bot to open multiple connections and process messages in parallel, so there is no point in implementing guaranteed delivery.
The other item to note is that we are implementing outbound messaging between our back end logic and the bot using Azure Service Bus. This allows our Discord bot to listen to a Service Bus queue, pickup messages and send them as soon as a message is added to the queue. Storage queues do not support code listening to the queue outside of the feature that allows Azure Functions to trigger off queued messages.
Adding Storage Queue Support
Firstly, as we are going to implement storage queues first, open up your package manager and add the Microsoft.Azure.Storage.Common and Microsoft.Azure.Storage.Queue packages to your project. We are then going to initialize a storage account in the OnStarted method of our Discord socket service by creating and adding a ConfigureStorageQueue method. This method will require the following class variables:
private CloudStorageAccount storageAccount to hold the storage account connection
Once these variables are created, add the following method to your Discord Socket Service class:
private void ConfigureStorageQueue()
{
try
{
storageAccount = CloudStorageAccount.Parse(_config["StorageQueueConnectionString"]);
}
catch (Exception ex)
{
_logger.LogError(ex.ToString());
return;
}
queueClient = storageAccount.CreateCloudQueueClient();
inboundQueue = queueClient.GetQueueReference("discord-bot-inbound-queue");
inboundQueue.CreateIfNotExistsAsync();
}
The first thing this method tried to do is create a storage account using a connection string. Rather than hard coding this, we need to have a valid connection string within the configuration options available to us. For the development environment, we can add another key/value pair to our hostsettings.json - "StorageQueueConnectionString": "UseDevelopmentStorage=true". This works under the assumption that you have the Azure Storage Emulator installed and running. Of course, if you are hosting this within Azure, already make sure your web host has an environment variable set with the production connection string set.
If we can successfully connect to the storage account, the method then creates a new queue client and tries to connect to the queue called discord-bot-inbound-queue in the specified account. For our development environment, we need to make sure that the queue has been created locally. To do this, in the view menu, select the Cloud Explorer side pane and expand the Local area. Under Local, there should be a Storage Account option with the Emulator underneath. In the Queue’s section, create a queue called discord-bot-inbound-queue.
If we run our bot now, you’ll see it still runs just fine but we don’t actually do anything with the queue we have just created.
Add Message to Queue
Before we start adding messages to the queue, we need to get it into a format we can use. To do this, we are going to create a helper class library that will take existing Discord entities and put them in a dummy object so we can convert it into a JSON string. The reason for this is because message queues can only take strings as the payload and the Discord entities aren’t constructed to convert easily. The class library allows us to make this a little more useful by sharing the library between projects.
Add a static class library as a new project to your solution. Remove the default class that is created as part of this project and create a new folder called Utils. In Utils, create a new static class called DiscordConvert we can use to convert objects with a SerializeObject method:
public static class DiscordConvert
{
public static string SerializeObject(SocketMessage message)
{
var converted = new ConvertedMessage(message);
return JsonConvert.SerializeObject(converted, Formatting.None);
}
}
The good thing about this class is we can overload it with different Discord event objects and return a JSON string that can be added onto the queue. This method uses a class we haven’t defined yet, ConvertedMessage, to take a small amount of the message so it can be serialized with JsonConvert. Let’s implement that now by creating a folder called Entities in our DNetUtils project and creating a ConvertedMessage class:
public class ConvertedMessage
{
public ulong AuthorId { get; set; }
public ulong ChannelId { get; set; }
public MessageSource Source { get; set; }
public string Content { get; set; }
public DateTimeOffset CreatedAt { get; set; }
public ICollection<ulong> MentionedChannelIDs { get; set; }
public ICollection<ulong> MentionedRoleIDs { get; set; }
public ICollection<ulong> MentionedUserIDs { get; set; }
public ConvertedMessage() { }
public ConvertedMessage(SocketMessage message)
{
AuthorId = message.Author.Id;
ChannelId = message.Channel.Id;
Source = message.Source;
Content = message.Content;
CreatedAt = message.CreatedAt;
MentionedChannelIDs = new List<ulong>();
MentionedRoleIDs = new List<ulong>();
MentionedUserIDs = new List<ulong>();
foreach (var channel in message.MentionedChannels)
{
MentionedChannelIDs.Add(channel.Id);
}
foreach (var role in message.MentionedRoles)
{
MentionedRoleIDs.Add(role.Id);
}
foreach (var user in message.MentionedUsers)
{
MentionedUserIDs.Add(user.Id);
}
}
}
We now have a function that will convert a Discord message and some of its attributes into a JSON string.
To implement this in our bot, we first need to add the new class library as a dependency reference to our bot project. Then, let’s modify the ReceiveMessage method to use this new class to take messages and put them on our queue. In the ReceiveMessage method, we are going to replace:
if (message.Content.ToLower().StartsWith("!ping"))
channel.SendMessageAsync("pong!");
with the following code:
CloudQueueMessage jsonMessage = new CloudQueueMessage(DiscordConvert.SerializeObject(message));
inboundQueue.AddMessage(jsonMessage);
This takes the JSON string produced by our serialize method and adds it to the inbound queue we created earlier.
Now, when a user posts a message in any of the channels our bot is in, the message is added to the storage queue for processing.
Process Queue with an Azure Function
The next step is to process use an Azure function to process the message and put a response onto a Service Bus queue. Firstly, let’s create a Service Bus Queue that we can use to put the return message on. At the time of writing this post, there is no way to run Service Bus locally so we will need to create one in Azure.
Create this service just using the basic plan, because at this stage, we only need to worry about the one service collecting messages so we don’t need any of the other advanced features like topics if we had multiple different messages types running through a single queue.
When the Service Bus instance has finished, create a new queue in the Service Bus call DNetBotQueue. This queue will be the default that our bot will monitor to pick up messages to send to various channels based on the processing that our functions do. We also need to add a Shared Access Policy.
Now we have a Service Bus queue ready to go, we need an Azure Function to pickup the message from our Storage Queue, “process it”, and send a response back to the user. At the moment, our processing will be just doing what we did previously, looking for the start of the text to say !ping and responding with a pong!. Firstly, create a new Azure functions project and name it appropriately. We are also going to remove the default Function class created when you first create the project and instead create a folder called Messaging. This just allows us to provide better structure when we want to add additional functions to the project to do other tasks.
Under this folder, we are going to create a new function with a Queue trigger. This will scaffold the new function for us with the correct method and dependencies all set up and ready to go.
Because we are also going to use this function to output to a ServiceBus queue, let’s also add the required Service Bus package. Once both of these items are installed, there are two things you will need to do. Firstly, let’s modify our hosts.json to make sure the following code exists:
"extensions": {
"queues": {},
"serviceBus": {}
}
This connects up the two required extensions at this point, ServiceBus and Storage Queues. Next, open the local.settings.json file and enter the following two lines:
"InboundMessageQueue": "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;
AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/
KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;
QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;",
"AzureWebJobsServiceBus": "endpoint-in-shared-access-key"
The first configuration parameter (InboundMessageQueue) is the default account and key for the storage emulator that we are using to monitor the storage queue for events. The second is the endpoint copied from the service bus Shared Access Key from your service bus queue.
Now that we have hooked up the accounts, let’s modify the Queue trigger we created to look something like this:
public static class ProcessMessages
{
[FunctionName("InboundMessageProcess")]
[return: ServiceBus("dnetbotmessagequeue", Connection = "AzureWebJobsServiceBus")]
public static string InboundMessageProcess
([QueueTrigger("discord-bot-inbound-queue")] CloudQueueMessage myQueueItem, ILogger log)
{
log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
ConvertedMessage message = DiscordConvert.DeSerializeObject(myQueueItem.AsString);
if(message.Content.StartsWith("!ping"))
{
var returnMessage = new NewMessage();
returnMessage.ChannelId = message.ChannelId;
returnMessage.Content = "pong!";
return JsonConvert.SerializeObject(returnMessage, Formatting.None);
}
return null;
}
}
If we step through this code, the first line lables the Function as InboundMessageProcess. The next line specifies that this function will through output to the ServiceBus service we have set up, specifically to the ‘dnetbotmessagequeue’ using the ‘AzureWebJobsServiceBus’ connection. We then define the method, taking our queue’s (discord-bot-inbound-queue) message and the logging system as inputs. After writing to the log, we then take or queue message and deserialize it back into a converted message. Finally, we process the message by seeing if it meets the criteria and crafting a response. The return from this method is then sent to the ServiceBus queue.
So with this function, there are two small, extra components we need to create - the Deserialize method and the NewMessage entity in the DNetUtils class library. Let’s create the Deserialize method:
public static ConvertedMessage DeSerializeObject(string jsonString)
{
return JsonConvert.DeserializeObject<ConvertedMessage>(jsonString);
}
Pretty easy, it just uses JSonConvert to deserialize into a specific object. Finally, the NewMessage object looks like this:
public class NewMessage
{
public ulong ChannelId { get; set; }
public string Content { get; set; }
}
Monitor the Return Queue
Now we are getting the messages processed and into our ServiceBus queue, let’s hook up our bot to monitor that queue and send messages back. Firstly, add the ServiceBus package to the DNetBot project and copy the same AzureWebJobsServiceBus from the local.settings.json file in the Azure Function we set up into the host.settings file in the DNetBot project. Next, we are going to add three new variables to the top of our DiscordSocketService:
private string serviceBusConnectionString;
const string QueueName = "dnetbotmessagequeue";
static IQueueClient servicebusClient;
These three variables store the Service Bus connection string, the queue name and the client object. Let’s retrieve the service bus connector string by retrieving the configuration in the constructor:
serviceBusConnectionString = _config["AzureWebJobsServiceBus"];
Next, we are going to add a new method that configures the service bus. In a similar way, we have a method to configure the Azure function. Firstly, the ConfigureServiceBus method:
private void ConfigureServiceBus()
{
servicebusClient = new QueueClient(serviceBusConnectionString, QueueName);
var handlerOptions = new MessageHandlerOptions(SBException)
{
MaxConcurrentCalls = 1,
AutoComplete = false
};
servicebusClient.RegisterMessageHandler(ProcessMessage, handlerOptions);
}
This method performs three key functions. Firstly, it setups the Service Bus client and queue based on the supplied configuration. Next, we configure the client with some default options, but we also link an exception handler (SBException) we need to write next. Lastly, we register a handler for messages using another function we need to write called ProcessMessage. This handler will take the Service Bus message, convert it and send the message back to the channel.
Let’s add the exception handler to catch the error in case something goes wrong:
private Task SBException(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
{
_logger.LogError("ServiceBus Error | Endpoint: "
+ exceptionReceivedEventArgs.ExceptionReceivedContext.Endpoint + " | "
+ exceptionReceivedEventArgs.Exception.Message);
return Task.CompletedTask;
}
Pretty simple method that logs the error out to our logging function. Finally, in the message events partial class, let's add the following method to send the message back to Discord:
private async Task ProcessMessage(Message message, CancellationToken token)
{
var bodyString = Encoding.UTF8.GetString(message.Body);
Formatter.GenerateLog(_logger, LogSeverity.Info, "Self",
"Sending message - Sequence: " +
message.SystemProperties.SequenceNumber +
" -- Message: " + bodyString);
try
{
NewMessage response = JsonConvert.DeserializeObject
(bodyString, typeof(NewMessage)) as NewMessage;
var channel = discordClient.GetChannel(response.ChannelId);
ITextChannel textChannel = channel as ITextChannel;
if (textChannel != null)
{
await textChannel.SendMessageAsync(response.Content);
}
else
{
Formatter.GenerateLog(_logger, LogSeverity.Error, "Self",
"Error sending message: Channel is not a text channel");
}
}
catch (Exception ex)
{
Formatter.GenerateLog(_logger, LogSeverity.Error, "Self",
"Error sending message: " + ex.Message);
}
await servicebusClient.CompleteAsync(message.SystemProperties.LockToken);
}
This method is fired when there is a message on the Service Bus that needs to be handled and firstly converts that message to a string (Service Bus messages are stored in a Byte Array format). Next, we try and convert the message to a NewMessage format and get the channel we are wanting to send a message to. We have to make sure this channel is capable of receiving text messages and, if it is, we then send the message to that channel using SendMessageAsync. Finally, we complete the Service Bus processing to ensure the message is removed from the Service Bus so it doesn’t get processed again.
Now when we run the bot, the messages should process from the Service Bus Queue.
Running Everything Together
Because we essentially now have two projects, an Azure Functions project and a Discord Bot project, let’s make the entire solution run together. Right click on the solution and under the Startup project option, select run multiple projects and select the bot and functions project to start. This will allow us to start both the bot project and the functions project and process messages from start to finish.
Summary
We now have a complete, running bot that offloads all messages to a storage queue which is then processed externally by an Azure Function. Then, if the message meets our requirements, we store another return message on a Service Bus queue. This queue is monitored by the bot and when the message comes in, the bot picks it up and sends it to the correct channel. The code based hosted here on GitHub has been updated to include the new projects and should be fully working.
Next post is probably going to be a little shorter and we will take a look at persisting messages to a storage account.
Stay tuned!
History
- 9th September, 2019: Initial version
