Dictating Text Messages with Plantronics and Twilio

• Download Fritz.DesktopSms.zip - 477.6 KB

I am a complete text message addict. There, I said it… I like to text my family and friends about all sorts of things that don’t need a full phone conversation. However, I hate typing on the little keyboard that my phone gives me, and it would be so much quicker if I could just dictate my messages.

Fortunately, with the help of Twilio and some cool API usage from my trusty Plantronics headset, I wrote a little app to connect these services and allow me to send text messages from my desktop with dictation capabilities. In this article, I will show you how I connected a Bluetooth headset, Microsoft Speech recognition, and Twilio to allow me to send text messages to my friends and family.

Configuration and Account Activation

Before I can start writing any code, I need to have the Plantronics SDK installed from the Plantronics Developer Connection site. After I signed up and installed the SDK for the headset, I went to Twilio and signed up for an account. If you want to follow along with this demonstration, you can sign up and stay in their trial period to see how this works. The only drawback is you will get some extra information sent in your text messages that indicate you are working with a trial account.

After choosing a phone number and completing the Twilio sign-up process, I was presented with the User Dashboard on Twilio that contained the information I would need to connect my application to the Twilio service.

We will require the Account SID and Auth Token values from this screen. Click the lock icon next to Auth Token to see the value of the token so that you can copy it for use in our application later.

A New Windows Forms Project

Starting with a new Windows Forms project, I can begin adding references and building out my utility application. I chose Windows Forms because I like the simplicity of the graphical interface and need access to the speech recognition libraries. I added a reference to my project for the Interop.Plantronics.dll file in the Program Files/Plantronics/SDK folder. If you have the Spokes 2.8 SDK installed, you need to right-click on the reference in Solution Explorer add set it to “Embed Interop Types = false” In future versions of the SDK, you will not have to set this option.

Now comes a secret helper library that is so cool, you have to use it just to make your life easier next time you program with the Plantronics headset. One of the Plantronics developers, Lewis Collins, wrote a C# wrapper class for their Spokes API to make interaction with the headset dead-simple. Download a copy of the SpokesWrapper file from GitHub and add it to the project. You won’t regret it.

Next, I can add a reference to the Twilio client by using NuGet and adding the Twilio package:

Finally, I added a reference to the System.Speech library in the .Net 4.5 framework. This library contains all of the functions necessary to enable speech recognition in this project. I had not used this before, and thought this was a great chance to enable speech functionality in one of my projects.

Enabling the Plantronics Headset

On the initial form in the project, I added a textbox named txtLog and docked it to the bottom of the screen. I will write log information to this box for the interactions with the headset. This way I can troubleshoot any rogue interactions that we not expected. I enable this log interaction be extending the form class with the Plantronics.UC.SpokerWrapper.DebugLogger interface. This interface adds a DebugPrint method that will be triggered each time an event happens on the headset device.

In the Form1_Load event handler, I added some code to get a reference to the headset and connect to it for the duration of the application:

public partial class Form1 : Form, DebugLogger
 {
  private Spokes _Spokes;
  private int _CallId = 1;

  private const string APP_NAME = "DesktopSms";

  public Form1()
  {
      InitializeComponent();

  }

  public void DebugPrint(string methodname, string str)
  {
      if (txtLog.InvokeRequired)
      {
        txtLog.Invoke(new Action<string,string>(DebugPrint), methodname, str);
        return;
      }
      txtLog.Text += string.Format("{2} {0}:\t{1}\r\n", methodname, str, DateTime.Now);
  }

  private void Form1_Load(object sender, EventArgs e)
  {

    this._Spokes = Spokes.Instance;
    _Spokes.SetLogger(this);

    _Spokes.Connect(APP_NAME);

  }

With the connection to the headset happening, and log information being output to the log textbox, I can focus on actually getting interactions with the headset.

I want to begin a dictation when I click a button on screen or press the button on the headset. I added a standard button to the screen and cleverly named it btnDictate. I also added a textbox called txtMyMessage to show the text of the message that was transcribed, also allowing me to make any changes before transmitting the message… I added an event handler for this, and connected the Spokes.ButtonPress event to trigger a DictateMessage method:

    // in Form1_Load
    _Spokes.ButtonPress += _Spokes_ButtonPress;

  }

  void _Spokes_ButtonPress(object sender, ButtonPressArgs e)
  {
    if (e.headsetButton == Interop.Plantronics.HeadsetButton.HeadsetButton_Talk)
    {
      DictateMessage();
    }

  }

  private void btnDictate_Click(object sender, EventArgs e)
  {
    DictateMessage();
  }

  private void DictateMessage()
  {
     txtMyMessage.Invoke(new Action(() => txtMyMessage.BackColor = Color.Yellow));
     _Spokes.OutgoingCall(++_CallId, "Sms Dictation");
  }

I changed the color of the txtMyMessage textbox to yellow so that I have a visual cue when the headset is ready to receive dictation. You will also notice that I issue a Spokes.OutgoingCall execution. This will “open the microphone” on the headset so that we can begin listening to the message.

The Simplest Speech Recognition You’ll Ever Need!

Speech recognition is something that I thought would be very hard to implement. Once I read through the documentation and a few CodeProject articles, I could tell that I would be able to implement this in just a few minutes.

I added a SpeechRecognitionEngine object to the Form class and initialized it in the Form constructor. In this way, I can be assured that the form is always ready for dictation. I added event handlers for the SpeechRecognized event and RecognizeCompleted event to allow me to capture the text that was transcribed and close the headset microphone.

  private SpeechRecognitionEngine _Recognizer;

  public Form1()
  {
    InitializeComponent();

    _Recognizer = new SpeechRecognitionEngine();
    _Recognizer.RequestRecognizerUpdate();
    _Recognizer.LoadGrammar(new DictationGrammar());
    _Recognizer.SpeechRecognized += _Recognizer_SpeechRecognized;
    _Recognizer.RecognizeCompleted += _Recognizer_RecognizeCompleted;

  }

The last piece of the engine that needs configuration is the Grammar. You’ll see that I called the LoadGrammar method and passed in the DictationGrammar, this is the default dictation grammar provided by Microsoft. It’s not perfect, and you can create your own grammar objects to meet your needs, but for me it was good enough.

My SpeechRecognized event handler is a simple trap of the text detected and writing it to the txtMyMessage textbox for review:

  void _Recognizer_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
  {
    DebugPrint("Detected Speech", e.Result.Text);
    txtMyMessage.Text = e.Result.Text;
    txtMyMessage.Invoke(new Action(() => txtMyMessage.BackColor = Color.White));
                
  }

I will shut off the Plantronics headset in the RecognizeCompleted event with this code:

  void _Recognizer_RecognizeCompleted(object sender, RecognizeCompletedEventArgs e)
  {
     _Spokes.EndCall(_CallId);
  }

Now, let’s proceed to actually send this message!

Transmitting a Message with Twilio

The final piece, transmitting the message with Twilio is embarrassingly easy to do. I’m ashamed to write up just how easy it is, because they’ve done a great job to provide an API that makes my life as a coder simple.

I added two constants to the top of this class that will hold my AccountSid and AuthToken that I recorded earlier from the Twilio user dashboard. With a new text to hold the destination phone number called txtDestination, I also added to my screen a button called btnSendMessage that will actually transmit the text in the txtMyMessage textbox. This is crazy simple… check out the event handler for this click event:

  private const string TwilioAccountSid = "MY ACCOUNT SID";
  private const string TwilioAuthToken = "MY AUTH TOKEN";
  private void btnSendMessage_Click(object sender, EventArgs e)
  {
    var client = new TwilioRestClient(TwilioAccountSid, TwilioAuthToken);
    var msg = client.SendSmsMessage("MY TWILIO NUMBER", txtDestination.Text, txtMyMessage.Text);
  }

I create a TwilioRestClient with the AccountSid and AuthToken that I defined earlier. With that client, I issue a SendSmsMessage with my Twilio phone number, the phone number to send to, and the text of the message to send. I love seeing APIs that are this simple to work with and use!

Summary

With these quick and easy methods I was able to connect my headset to a text messaging platform. This sample project could use some validation of message content and length. If I wanted, I could possibly be integrate the functionality into a larger application. For my needs, I can send funny text messages to my wife without touching my phone and using my cool Plantronics headset. I have attached my source code to this project for your reference if you would like to try it for yourself.

This article originally appeared on the Plantronics Developer Zone.