Introduction
I was last year creating an UWP fun app, Play Your Solo, for Window Store. For that I needed a software keyboard with different instrument in the app. I search the net for a tutorial and code examples but didn't found anything up to date for C#.
As a kind of a beginner to programming in C# and using Visual Studio, finishing the app would have been done faster, if I could have done some copy/paste from a good example. Although it was not very complicated to do and I learned a lot from creating it myself.
For people looking for a simple way to create a software MIDI keyboard as part of an app, or to use as a startup training project into the MIDI world, I created this little example. It does not have a finished touch, as the XAML part is not that interesting.
Background
As a beginner to using MIDI format and using the built in Microsoft GS Wavetable Synth, I found a document on Windows Dev Center, MIDI, that gave me the basic setup of the program. That document is easy read and understood.
For a better understanding of MIDI messages, I found this page by Dominique Vandenneucker from Carnegie Mellon Univerity, School of Computer Science. For the instrument used in the General MIDI (GM), this page was very useful. It is a handsout from a music course at Center for Computer Assisted Research in the Humanities at Stanford University.
Using the code
First of all, start up Visual Studio 2015. The Windows 10 SDK has to been installed. Create a New Project and choose the template "Blank App (Universal Windows)". Call the project MidiKeyBoard.
When the project has been built, you need to reference the SDK extension for the built-in MIDI synth. To do so, right-click on 'References' in the Solution Explorer, in Visual Studio, and choose 'Add Reference'.
A new window will pop-up. In that window, expand the 'Universal Window'-node and click on 'Extensions'.
Depending on your configuration of Visual Studio, a longer list of available extensions will show. Select the extension 'Microsoft General MIDI DLS for Universal Windows Apps'.
If there are more than one extension with that name, choose the version, that matches the target of your app. To check the target version of the app, go to the project properties and select the Application tab.
Click 'OK' and the extension is added
Now that the SDK extension for the built-in MIDI synth is added, it is time to do some simple layout of the MIDI keyboard. First of all, lock the app in Landscape mode, as the keyboard will look funny and small in Portrait mode.
To do so, dobble-click the Package.appxmanifest in the Solution Explorer. Select the Application tab and check Landscape only. This will lock the screen in Landscape mode.
In MainPage.xaml, create three rows in the main grid. One for status line, one for the keyboard and one for choosing instruments.
Set the background color of the grid to yellow, to be able to see the black and white key og the keyboard
The status will consist of at TextBlock to show if any MIDI device is connected. This should look like this:
<Page
x:Class="MidiKeyboard.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:MidiKeyboard"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid Background="Yellow">
<Grid.RowDefinitions>
<RowDefinition Height="Auto"/>
<RowDefinition Height="*"/>
<RowDefinition Height="Auto"/>
</Grid.RowDefinitions>
<TextBlock Name="StatusTextBlock"
Grid.Row="0"
Margin="10" />
</Grid>
</Page>
In code behind, MainPage.xaml.cs, add the namespaces Windows.Devices.Enumeration and Windows.Devices.Midi, to be able to use enumaration of devices and MIDI devices. Also add the System.Threading.Tasks namespace to be able to do Task in async methods
...
using Windows.Devices.Enumeration;
using Windows.Devices.Midi;
System.Threading.Tasks;
...
To find the Microsoft GS Wavetable Synth to connect to it, enumerate through all the MIDI Output devices connected to the Windows 10 device. This is done by the FindAllAsync method of the DeviceInformation class. Create a private method EnumerateMidiOutputDevices().
In the EnumerateMidiOutputDevices() method, create a string to hold the query information for the FindAllAsync method. The query will hold information to get all MIDI output ports connected. The query string will be used as input for the FindAllAsync method and the search result will be gathered in a DeviceInformationCollection object
If no MIDI output devices is found, return from the method and set the text in the StatusTextBlock to "No MIDI output devices found". If a MIDI output device, hopefully the Microsoft GS Wavetable Synth, the name will be displayed in the StatusTextBlock. In this example it is assumed, that no external MIDI output devices are connected, so the collection of MIDI output devices will only have one item. The one needed, Microsoft GS Wavetable Synth.
As EnumerateMidiOutputDevices() is an async method returning a Task, it should not be called directly from the MainPage(). Create a method, GetMidiOutputDevicesAsync(), to call EnumerateMidiOutputDevices(). GetMidiOutputDevicesAsync() should be called form MainPage(). It will look like this.
MainPage method:
public MainPage()
{
this.InitializeComponent();
GetMidiOutputDevicesAsync();
}
GetMidiOutputDevicesAsync:
private async void GetMidiOutputDevicesAsync()
{
await EnumerateMidiOutputDevices();
}
EnumerateMidiOutputDevices method:
private async Task EnumerateMidiOutputDevices()
{
string midiOutportQueryString = MidiOutPort.GetDeviceSelector();
DeviceInformationCollection midiOutputDevices = await DeviceInformation.FindAllAsync(midiOutportQueryString);
if (midiOutputDevices.Count == 0)
{
StatusTextBlock.Foreground = new SolidColorBrush(Colors.Red);
StatusTextBlock.Text = "No MIDI output devices found";
return;
}
else
{
StatusTextBlock.Foreground = new SolidColorBrush(Colors.Green);
StatusTextBlock.Text = midiOutputDevices[0].Name;
}
}
Now that the MIDI output device, hopefully Microsoft GS Wavetable Synth, is found, create an instance of DeviceInformation to hold information of Microsoft GS Wavetable Synth. That will be used to create an object of the interface IMidiOutPort using the Id property of the DeviceInformation. This object will be used to send the MIDI message to be effectuated. Due to that, define the IMidiOutPort object in the top of the code-behind for the page, so it can be used outside the EnumerateMidiOutputDevices method.
Top of the code behind of the page:
...
namespace MidiKeyboard
{
public sealed partial class MainPage : Page
{
IMidiOutPort midiOutPort;
public MainPage()
{
...
In the EnumerateMidiOutputDevices:
...
else
{
StatusTextBlock.Foreground = new SolidColorBrush(Colors.Green);
StatusTextBlock.Text = midiOutputDevices[0].Name;
}
DeviceInformation devInfo = midiOutputDevices[0];
if (devInfo == null)
{
StatusTextBlock.Foreground = new SolidColorBrush(Colors.Red);
StatusTextBlock.Text = "No device information of MIDI output";
return;
}
midiOutPort = await MidiOutPort.FromIdAsync(devInfo.Id);
if (midiOutPort == null)
{
StatusTextBlock.Foreground = new SolidColorBrush(Colors.Red);
StatusTextBlock.Text = "Unable to create MidiOutPort from output device";
return;
}
}
When debugging now, you can find the Id property for the Microsoft GS Wavetable Synth and hardcode the Id in the FromIdAsync method. The enumeration with the FindAllAsync method could then be avoided everytime the applications is loaded. An disavantage of that is, that the application will stop working, if the Id somehow changes from device to device or with an update.
To release resources when the app is suspended, create a methods to be called, when the app is suspended. In this method, call the dispose method for the MidiOutPort and set the MidiOutPort to null
private void App_Suspending(object sender, SuspendingEventArgs e)
{
try
{
midiOutPort.Dispose();
midiOutPort = null;
}
catch
{
}
}
When the application is resumed after suspension, the connection to the MIDI output port should be reestablished. Create a method to call the EnumerateMidiOutputDevices() method to set up the MIDI connection.
private async void App_Resuming(object sender, object e)
{
await EnumerateMidiOutputDevices();
}
As a final step, add the eventhandlers to the event of suspending and resuming in the MainPage().
...
public MainPage()
{
this.InitializeComponent();
GetMidiOutputDevicesAsync();
Application.Current.Resuming += new EventHandler<Object>(App_Resuming);
Application.Current.Suspending += new SuspendingEventHandler(App_Suspending);
}
...
Now that the inital setup for using the Microsoft GS Wavetable Synth has been made, the layout has to be set. This layout will be very simple and not very nice looking. A XAML code can easely be very voluminous and not easely read, when creating a good layout. For the same reason, only five keys will be showed, chromatically from C to E.
In the MainPage.xaml, create a grid element inside the main grid, named KeyboardGrid, and set it to grid row 1 of the main grid.
In KeyboardGrid, define five columns to hold the five keys in the keyboard. Make five rectangles, placed in each column. Let the name of each rectangle be the name of the key: Ckey, CSharpKey, DKey, DSharpKey, EKey. Set the fill color to alternate white and black
In each rectangle elements create two events to control the keyboard. Create an event for PointerPressed to call the Key_PointerPressed eventhandler, when a key is pressed. Create an event for PointerReleased to call the Key_PointerReleased eventhandler, when a key is released again.
The grid, KeyboardGrid, should look like this:
...
<Grid Background="Yellow">
<Grid.RowDefinitions>
<RowDefinition Height="Auto"/>
<RowDefinition Height="*"/>
<RowDefinition Height="Auto"/>
</Grid.RowDefinitions>
<TextBlock Name="StatusTextBlock"
Grid.Row="0"
Margin="10" />
<Grid Name="KeyboardGrid"
Grid.Row="1"
Margin="20">
<Grid.ColumnDefinitions>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
</Grid.ColumnDefinitions>
<Rectangle Name="CKey"
Grid.Column="0"
Fill="White"
PointerPressed="Key_PointerPressed"
PointerReleased="Key_PointerReleased"/>
<Rectangle Name="CSharpKey"
Grid.Column="1"
Fill="Black"
PointerPressed="Key_PointerPressed"
PointerReleased="Key_PointerReleased"/>
<Rectangle Name="DKey"
Grid.Column="2"
Fill="White"
PointerPressed="Key_PointerPressed"
PointerReleased="Key_PointerReleased"/>
<Rectangle Name="DSharpKey"
Grid.Column="3"
Fill="Black"
PointerPressed="Key_PointerPressed"
PointerReleased="Key_PointerReleased"/>
<Rectangle Name="EKey"
Grid.Column="4"
Fill="White"
PointerPressed="Key_PointerPressed"
PointerReleased="Key_PointerReleased"/>
</Grid>
</Grid>
...
Hopefully, the application should look something like this, when it is set to run in Visual Studio 2015:
In the eventhandler for Key_PointerPressed, a MIDI message will be send to play a specific note. A way to communicate with the MIDI synthesizer is through the IMidiMessage interface. In this app, we will be using three type of messages in the interface, Note On, Note Off and Program Change. The General Midi (GM) standard specify 16 channels and 128 predefined instrument. As the MIDI keyboard only will play with one instrument at the time, only channel 0 will be used in this example.
The Program Change message is used to select or change the instrument of the MIDI output device in a specific channel. The message is like this: MidiProgramChangeMessage(byte 'Channel Number', byte 'Instrument Number'). In this example, we use channel 0 and set the instrument to Acoustic Grand Piano wich is the first instrument in the list of predefined instrument. As we use the list in a program, we will substract 1 from the number in the list to get the correct instrument. Now define the IMidiMessage in the head of the code-behind of the page and create the MidiProgramChangeMessage in the MainPage().
It should look like this:
...
public sealed partial class MainPage : Page
{
IMidiOutPort midiOutPort;
IMidiMessage instrumentChange;
public MainPage()
{
this.InitializeComponent();
instrumentChange = new MidiProgramChangeMessage(0, 0);
GetMidiOutputDevicesAsync();
Application.Current.Resuming += new EventHandler<Object>(App_Resuming);
Application.Current.Suspending += new SuspendingEventHandler(App_Suspending);
}
...
To set the initial instrument, send the created midi message, instrumentChange, to the MidiOutPort, midiOutPort, in the end of the EnumerateMidiOutputDevices() method:
...
if (midiOutPort == null)
{
StatusTextBlock.Foreground = new SolidColorBrush(Colors.Red);
StatusTextBlock.Text = "Unable to create MidiOutPort from output device";
return;
}
midiOutPort.SendMessage(instrumentChange);
}
...
Now that the instrument of the MIDI output device is set, the keyboard has to be made functional. To start a MIDI sound from the chosen instrument with a chosen frequency (pitch), send the MIDI message Note On. To stop the sound again, send the MIDI message Note OFF.
The range of the pitch goes from 0 to 127. The middle C has the value 60, and the pitch step a half note. That results in D# has the value 61 and D has the value 62 etc.
As with real instrument, some instruments hold the sound (maybe with vibrating or variation) as long as the notes are being played, like the trumpet. Some instruments fade the sound as the notes are being played, like the piano.
The velocity of a MIDI sound define the power in the sound. Higher number has more power. Velocity range from 0 to 127. In this example we use the velocity 127.
A Note On message has the format: MidiNoteOnMessage(byte 'Channel Number', byte 'Note', byte 'Velocity')
Similiar, when we want to stop a specific note in a specific channel, we use the MIDI message Note Off. A Note Off message has the format: MidiNoteOffMessage(byte 'Channel Number', byte 'Note', byte 'Velocity')
As we in this example always use the channel 0 and a velocity of 127, let them be created in the top of the code-behind for the page.
...
public sealed partial class MainPage : Page
{
IMidiOutPort midiOutPort;
IMidiMessage instrumentChange;
byte channel = 0;
byte velocity = 127;
public MainPage()
{
this.InitializeComponent();
...
In the eventhandler for PointerPressed, Key_PointerPressed, we will extract the key pressed using a switch/case and select the pitch according to that. When the pitch is knowned, the Note On message can be created with the channel and velocity and the message can be send to the MIDI output device. To be able to create a Rectangle element, to cast the pressed key, add the namespace Windows.UI.Xaml.Shapes.
The Key_PointerPressed will look like:
private void Key_PointerPressed(object sender, PointerRoutedEventArgs e)
{
byte note;
Rectangle keyPressed = (Rectangle)sender;
string keyPressedName = keyPressed.Name;
switch (keyPressedName)
{
case "CKey":
note = 60;
break;
case "CSharpKey":
note = 61;
break;
case "DKey":
note = 62;
break;
case "DSharpKey":
note = 63;
break;
case "EKey":
note = 64;
break;
default:
note = 60;
break;
}
IMidiMessage midiMessageToSend = new MidiNoteOnMessage(channel, note, velocity);
midiOutPort.SendMessage(midiMessageToSend);
}
To turn off the specific note again, the same code is used in the eventhandler Key_PointerReleased. Only this time creating a Note Off message to send to the MIDI output device.
private void Key_PointerReleased(object sender, PointerRoutedEventArgs e)
{
byte note;
Rectangle keyPressed = (Rectangle)sender;
string keyPressedName = keyPressed.Name;
switch (keyPressedName)
{
case "CKey":
note = 60;
break;
case "CSharpKey":
note = 61;
break;
case "DKey":
note = 62;
break;
case "DSharpKey":
note = 63;
break;
case "EKey":
note = 64;
break;
default:
note = 60;
break;
}
IMidiMessage midiMessageToSend = new MidiNoteOffMessage(channel, note, velocity);
midiOutPort.SendMessage(midiMessageToSend);
}
If the applictation is made to run in Visual Studio 2015, the MIDI keyboard should now be functionally, both for computers using mouse or touchpad and for devices using touch screen.
The MIDI device is able to play more than one note at the time with different pitch. Therefore the keyboard created will be able to play polyphonic sounds. Just press more than one key at the time to create harmonies.
It would be more useful to be able to select different instruments. To do so, in the XAML page, create a StackPanel in the main grid in the third row. The StackPanel should have a horizontal orientation. Inside the StackPanel, create four buttons to be able to select different instruments. Each buttons should call the same eventhandler, when clicked.
The additional code in the XAML page could look like:
...
<StackPanel Grid.Row="2"
Orientation="Horizontal"
Margin="20">
<Button Name="Piano"
Margin="0,0,10,0"
Content="Piano"
Click="Selection_Click"/>
<Button Name="Trombone"
Margin="0,0,10,0"
Content="Trombone"
Click="Selection_Click"/>
<Button Name="Trumpet"
Margin="0,0,10,0"
Content="Trumpet"
Click="Selection_Click"/>
<Button Name="Flute"
Margin="0,0,10,0"
Content="Flute"
Click="Selection_Click"/>
</StackPanel>
...
Before the Selection_Click eventhandler is created, some theory of instruments has to be informed. Most instruments have a natual interval of frequency, where it is useful. The interval depends upon the physical shape of the part of the instrument, that amplifies the sound. It does not sound good to play very high notes on a bass. Likewise, you will never play a low note on a picolo flute. The piano is special, because of the wide range of frequecy, but in this example it is set to be the same as the trumpet.
In the present example the Trombone has a frequency range, one octave lower than the trumpet and piano. The frequency range of the flute is one octave higher than the trumpet and piano.
One octave in a MIDI device is equal to 12 steps in value of the pitch. To contol the octave interval an integer is defined in the top of the page in the code-behind.
...
public sealed partial class MainPage : Page
{
IMidiOutPort midiOutPort;
IMidiMessage instrumentChange;
int octaveInterval = 0;
byte channel = 0;
byte velocity = 127;
public MainPage()
{
...
In the button eventhandler Selection_Click, create a button element to get the name of the selected button. The name of the button will be use to select a new MIDI instrument number and frequency interval using a switch/case. When the instrument number is selected, a new Program Change message is created and sent to the MIDI output device.
The Selection_Click() eventhandler should look like:
private void Selection_Click(object sender, RoutedEventArgs e)
{
byte midiInstrument;
Button instrumentNameButton = (Button)sender;
string instrumentName = instrumentNameButton.Name;
switch (instrumentName)
{
case "Piano":
midiInstrument = 0;
octaveInterval = 0;
break;
case "Trombone":
midiInstrument = 57;
octaveInterval = -1;
break;
case "Trumpet":
midiInstrument = 56;
octaveInterval = 0;
break;
case "Flute":
midiInstrument = 73;
octaveInterval = 1;
break;
default:
midiInstrument = 0;
octaveInterval = 0;
break;
}
instrumentChange = new MidiProgramChangeMessage(channel, midiInstrument);
midiOutPort.SendMessage(instrumentChange);
}
The final step is to let the pitch of the played note be depended of the instrument frequency interval set in the eventhandler of the buttons to select instrument. This is done by adding a part to the picth selection in the switch/case in Key_PointerPressed and Key_PointerReleased eventhandler. The part is simply '12 * octaveInterval' to either raise or lower the frequency interval by an octave.
The Key_PointerPressed eventhandler now should look like:
...
Rectangle keyPressed = (Rectangle)sender;
string keyPressedName = keyPressed.Name;
switch (keyPressedName)
{
case "CKey":
note = (byte)(60 + (octaveInterval * 12));
break;
case "CSharpKey":
note = (byte)(61 + (octaveInterval * 12));
break;
case "DKey":
note = (byte)(62 + (octaveInterval * 12));
break;
case "DSharpKey":
note = (byte)(63 + (octaveInterval * 12));
break;
case "EKey":
note = (byte)(64 + (octaveInterval * 12));
break;
default:
note = (byte)(60 + (octaveInterval * 12));
break;
}
IMidiMessage midiMessageToSend = new MidiNoteOnMessage(channel, note, velocity);
...
Likewise the Key_PointerReleased eventhandler should look like:
...
Rectangle keyPressed = (Rectangle)sender;
string keyPressedName = keyPressed.Name;
switch (keyPressedName)
{
case "CKey":
note = (byte)(60 + (octaveInterval * 12));
break;
case "CSharpKey":
note = (byte)(61 + (octaveInterval * 12));
break;
case "DKey":
note = (byte)(62 + (octaveInterval * 12));
break;
case "DSharpKey":
note = (byte)(63 + (octaveInterval * 12));
break;
case "EKey":
note = (byte)(64 + (octaveInterval * 12));
break;
default:
note = (byte)(60 + (octaveInterval * 12));
break;
}
IMidiMessage midiMessageToSend = new MidiNoteOffMessage(channel, note, velocity);
...
Now the simple MIDI software keyboard example is finished.
Points of Interest
A thing to remember from this project is, how simple and easy it is to use the MIDI format and MIDI devices. This was a bit scary to me, before I started creating a MIDI software keyboard. A way ahead for a new project will be to make an application that can create MIDI melodies with more than one channel in use, and to be able to play then also. This seems a bit more complicated, but hopefully it will turn out to be as easy as creating Note On/Note Off message and send them to a MIDI output device.
History
- Creation of the initial article and demo source code 20170228
