Using OpenTK/OpenAL to Develop Cross Platform DIS VOIP Application

• Download source code - 2.23 MB

Introduction

The intent of this application is to provide an example on how OpenAL (via OpenTK) can be used for voice communication utilizing the Distributed Interactive Simulation (DIS) IEEE Standard 1278.1 while running under Mono. The motivation for writing this article was due to my quest to find examples for both of these technologies. As there were not many or any examples found on the internet in using either OpenAL (under the .NET Framework) and DIS, I felt that providing this example might be helpful for other developers. Note that this technology is similar to using Voice Over Internet Protocol (VOIP) but does not use "session control" to handle calls, therefore this application is just voice encoded data being transmitted over the internet.

Background

Open Audio Library (OpenAL) is a cross-platform 3D audio API that several game developers have used successfully in the past. The Open Tool Kit (OpenTK) developers provided a wrapper around the OpenAL API (along with OpenGL and OpenCL) which then provides the functionality to work with any Mono/.NET language.

Distributed Interactive Simulation (DIS) or IEEE 1278.1 has been around since 1995 and is mainly used by military organizations for conducting simulations. This protocol was chosen due to the work done by the MOVES institute at the Naval Post Graduate School in developing the Open-DIS implementation. As a contributor to that project, for the C# effort, my intent was to capitalize on the work done and create a product encompassing both OpenAL and Open-DIS.

Code Description

The code provided shows only one of the ways in which the OpenAL API can be used to capture and transmit audio over the internet. There are four modules and one main form application.

• AudioIn: Project that contains all necessary methods to Initialize, Start and Stop the microphone
• AudioOut: Project that contains all necessary methods to Initialize and Play audio data
• DISNET: Project that contains methods used to process DIS Protocol Data Units (PDU), in particular the Transmitter and Signal PDU
• Sockets: Project that contains the socket communication architecture. The implementation provided will use UDP broadcasting only.
• OpenDISRadioTransmitter: Project that contains the main form which encapsulates all the other classes to produce a simplistic radio interface. This form/application is the 'Radio' when referenced in the description below. A Radio in this case will have One Transmitter and multiple Receivers. Note that a simplistic GUI interface was designed on purpose as the main goal of this project was to introduce OpenAL and DIS.

RadioTestSuite Form

This is the main form for generating the GUI and the start point for all initializations. From the RadioTestSuite constructor, all initializations are started.

public RadioTestSuite()
{
	InitializeComponent();
	InitializeTimer();
	InitializeRadioCommunications();
	InitializeRadioReceiversTransmitters();
}

InitializeTimer()

This method starts a StopWatch that will be used to set the timestamp of all outgoing PDUs. The timestamp data is not used in this application and only provided for completeness.

InitializeRadioCommunications()

This method sets up the sockets for UDP Broadcast Receive and Transmit. A random port of 9737 was chosen. In this method, the Transmitter properties are provided default settings. In DIS, the Transmitter PDU contains the frequency of the Radio transmitting and whether or not the radio is on, on and Transmitting, or on and not Transmitting. This information is stored in a collection by each receiving radio to determine if the audio should be played (if it matches a receiver set to the same frequency). As each radio should be unique and have only one transmitter, the Entity ID was chosen to contain the last octet of the IP Address of the system it is running on. Therefore if two of these applications are running on the same machine, there will be conflict unless the RadioID or EntityID is changed. The Transmitter PDU is broadcast out every 5 seconds (heartbeat) to allow all receivers to update the status of that radio transmitter.

The Signal PDU also has some default values set. In DIS, the Signal PDU contains the actual digital audio voice data. The Signal PDU contains the sample rate, number of samples and the encoding type along with the encoded audio. There are several types of encoding schemes outlined in the IEEE standard, but in this example application only uLaw (implementation provided by LumiSoft) was used.

Finally the microphone and the audio out features are initialized. The Microphone is set to read in Mono16 therefore 2 bytes of data will be read in at a time which accounts for the buffer size being double the number of samples.

StartMicrophone(audioSamplingRate, microphoneGain,
AudioCapture.DefaultDevice, ALFormat.Mono16, numberOfSamples * 2); 

InitializeRadioReceiversTransmitters()

This method scans through the form controls until it finds a usercontrol that matches 'Radio'. Once found, a uniqueID and Name are provided. This uniqueID will be used to identify which Radio control was pressed to set up the frequency and speaker selection.

Microphone

The method OutputMicrophoneSoundToSignalPDU() is a separate thread which is used to poll the microphone for data. This method was chosen to alleviate the overhead of stopping and starting the microphone. In this method, if the Transmit button on the form is pressed (isPTTActivated = true) a Transmit PDU is sent out specifying that the "TransmitterOnTransmitting" and then any microphone data that is collected is sent out over a Signal PDU. A Queue is used to collect any microphone data.

private void OutputMicrophoneSoundToSignalPDU()
{
	while (continueSendingMicrophoneData)
	{
		if (microphone.MicrophoneData.Count > 0)
		{
			//Remove data from the Queue
			byte[] raw = microphone.MicrophoneData.Dequeue();

			//If PTT is true then data will be sent out on the PDU
			if (isPTTActivated == true)
			{
				byte[] encodedData = uLaw.Encode(raw, 0, raw.Length);
				raw = uLaw.Decode
					(encodedData, 0, encodedData.Length);
				signalPDU.ExerciseID = transmitterPDU.ExerciseID;
				signalPDU.EntityId = transmitterPDU.EntityId;
				signalPDU.RadioId = transmitterPDU.RadioId;
				signalPDU.Data = encodedData;
				signalPDU.Samples = System.Convert.ToInt16
							(encodedData.Length);
				signalPDU.Timestamp = 
					TimeStamp(TypeTimeStamp.Absolute);
				DISnet.DataStreamUtilities.DataOutputStream ds2 =
				new DISnet.DataStreamUtilities.DataOutputStream
				(endianType);
				signalPDU.marshalAutoLengthSet(ds2);
				sendData.BroadcastMessage(ds2.ConvertToBytes());
			}
		}

		Thread.Sleep(1);
	}
}

Audio Out

The audio out portion of code occurs when a packet is received from the socket. If it is a Transmitter PDU, then the appropriate frequency and IDs are stored in a collection. When a subsequent Signal PDU comes in and its ID and frequency are compared to that of the Transmitter collection, if there is a match then the audio is played back. In the case of this application, the users can select which speaker to output the sound. This allows for two different conversations to take place (left or right).

//Play back unencoded data
playAudio.PlayBackAudio(unEncodedData, ALFormat.Mono16,
(int)signalPDU.SampleRate, retrievedFreqSpeakerTransmitterReceiver.SpeakerLocation);

The actual playing back of the audio is provided by the OpenAL API which is listed in the following code snippet. This sets up a unique int to a buffer using AL.GenBuffer(). This buffer is then used to copy the data (unencodedData) to that buffer using AL.BufferData(....). The speaker position is then set using ALSource3f.Position (note that if using a surround sound speaker system vice headset, there might be some sound coming out of the center speaker due to the 3D nature of OpenAL). After the buffer is loaded, the audio is played back and buffers are cleared out.

/// <summary>
/// Playback the audio
/// </summary>
/// <param name="unencodedData">Raw byte data</param>
/// <param name="recordingFormat">OpenAL sound format</param>
/// <param name="sampleFrequency">Frequency of the samples</param>
/// <param name="speakerLocation">Speaker location</param>
public void PlayBackAudio(byte[] unencodedData, ALFormat recordingFormat,
	int sampleFrequency, SpeakerLocation speakerLocation)
{
	//Determine if sources needed to be switched
	if (sourcesLeft == 0)
	{
		sourcesLeft = sources.Length;
	}

	//Used to rotate the sources being used.
	sourcesLeft--;

	int buf = AL.GenBuffer();

	AL.BufferData(buf, recordingFormat, unencodedData, 
			unencodedData.Length, sampleFrequency);

	position = SetSpeakerPosition(speakerLocation);
	AL.Source(sources[sourcesLeft], ALSource3f.Position, ref position);

	AL.SourceQueueBuffer(sources[sourcesLeft], buf);
	if (AL.GetSourceState(sources[sourcesLeft]) != ALSourceState.Playing)
	{
		ClearSourcePlayBackBuffers(sources[sourcesLeft]);
		AL.SourcePlay(sources[sourcesLeft]);
	}

	ClearSourcePlayBackBuffers(sources[sourcesLeft]);
}

Using the Application

Prior to using, the following library OpenAL must be installed. For Windows, use the one titled "OpenAL installer for Windows". When I tested with SUSE Linux, there was an issue/bug with the version provided 1.10.622, so the steps that I took were to install the OpenAL from SUSE's YAST and then download the following OpenAL source code version 1.11.753 and followed the instructions on that site for compiling. After compiling, overwrite the binaries that Linux installed. The version 1.11.753 or greater is needed for it to work on Linux. The OpenTK does not need to be installed as the binary is provided with the source code of this project. But for your own development, there are examples that come with the download. For the development environment, either the Windows Framework (version 3.5 or higher) or the Mono-Project framework 2.6.1 or higher needs to be installed. To compile the source, either use a Windows IDE (Express or Professional) or MonoDevelop version 2.2.1 or higher. Note that MonoDevelop works with both Windows and Linux and was used for my openSUSE testing. Once these are installed, then the test application provided can be compiled and run. There is an issue with using a USB headset with Linux and OpenAL. There are some headsets that will not be recoginzed so it appears not to be working. A search on the internet might turn up a solution. I had this problem with Open SUSE and I decided to use the onboard audio card instead. I did not have this problem with OpenAL running on Windows.

The following are screen shots and information on how to use the test application provided. This screen shows three Radio controls, a Transmit button and a check box for audio loop back. The bottom status bar shows the broadcast IP address being used for transmit and receive, along with the count of the number of packets received.

Single click the first Radio control (shown as a medium gray box surrounded by a light gray box). Once clicked, the following will pop up:

For this test case, put a checkmark in the Transmit/Receive box. This will set this first Radio Control to be a Transmitter and Receiver. Then enter a Frequency, use 1 for this example, in the box provided. Select Apply. Once completed, the main form will display the frequency entered.

At this point, put a check mark in the Loop Back Audio checkbox. This will allow any audio transmitted to be played back to the speaker. Press the TRANSMIT button and talk into the microphone, audio should be heard from the speaker.

The color indicators are as follows:

• The BaseBackgroundShadowColor is set to Transparent to indicate that this Radio Control is neither a Transmitter or Receiver.
• The ReceiverColor is set to DarkBlue which indicates that the Radio Control is a Receiver Only.
• The ReceivingSignalColor is set to Yellow which is the font color of the frequency displayed when receiving a transmission.
• The TransmitterColor is set to Red which indicates that the Radio Control is a Transmitter and Receiver.
• The TransmittingColor is set to Maroon which is the background color of the Radio Control when Transmitting.

NOTES

My goal was to provide a quick example of how to implement voice playback using the OpenAL API. There are a lot of other features of OpenAL that were not covered, but this example should at least start you down the path. Also this application was developed using Visual Studio 2008, as a result the Windows Forms do not appear in MonoDevelop (maybe in the near future) therefore any development in using that product should use the GTK# Visual Designer.

During testing of the audio, I enabled the Push-To-Talk to always be transmitting. After a few days, I noticed that on the Windows system the audio buffer was significantly lagging, such that when someone spoke on the Linux transmitter it would be heard many minutes later on the Windows box. In similar applications, I have developed for just Windows using the WaveInOpen (Win32 Audio) this problem was also there. As there most likely will never be an active transmitter on all the time, I do not foresee a problem, however, as a fix I created two sources and switch the incoming audio between them. This allows the buffer time to clear and as the data coming in is serial (do to the nature of the sockets and locking of incoming data) the chance that one audio packet being played before its predecessor should not happen.

I was never able to test a USB Headset with the Linux setup due to the issues outlined above with OpenAL. There are fixes outlined in searches I had done but as I am not an expert in Linux I decided to just use the onboard sound card, which worked great. I was only able to test with three systems (two windows and one Suse Linux) so I do not know how well this software scales.

History

• 28^th February, 2010: Initial post
• 12^th March, 2010: Source code updated