Audacity* Gets a Touch UI for Tablets and 2 in 1 PCs

Intel® Developer Zone offers tools and how-to information for cross-platform app development, platform and technology information, code samples, and peer expertise to help developers innovate and succeed. Join our communities for the Internet of Things, Android, Intel® RealSense™ Technology and Windows to download tools, access dev kits, share ideas with like-minded developers, and participate in hackathon’s, contests, roadshows, and local events.

Related articles:
Developer's Guide for Intel® Processor Graphics for 4th Generation Intel® Core™ Processors
Touch and Sensors
How to Create a Usable Touch UI
How to Adjust Controls for Touch

Introduction

This article previews some in-development features of Audacity, which is a free, simple, multi-track audio editor and recorder for Windows*, Mac* OS X*, GNU/Linux*, and other operating systems. The prototype adds a touch user interface (UI), providing the operator a simplified screen with the most frequently used commands and tools. In addition to providing a useful UI for running Audacity on a tablet, this enhanced version also detects the transition between tablet and laptop mode in Ultrabook™ 2 in 1s, then adjusts the display appropriately. It also provides a manual method for transitioning the user interface between these two modes.

Audacity in a 2 in 1

Audacity is an Open Source sound editing program with production-quality, audio engineering-level tools. It was created in 1999 by Carnegie-Mellon grad student Dominic Mazzoni and his advisor Roger Dannenberg and has been updated continuously ever since. Audacity has won numerous awards for its ease of use and powerful functionality including a BOSSIE (Best of Open Source Software) for sound editing and SourceForge Community Choice Awards for Best Project for Multimedia. An established PC program, Audacity’s user interface was designed around mouse and keyboard. With the popularity of touch-enabled PCs and Ultrabook 2 in 1s, Audacity’s UI was modified to take advantage of touch input and the 2 in 1s’ special capabilities. This article details the challenges and solutions discovered by software engineers in adding these capabilities to Audacity.

Many hobbyist-level musicians have learned the ropes of audio recording and editing using Audacity while sitting at their home computers. Navigating traditional keyboard and mouse controls on a laptop or desktop computer is a challenge while playing a guitar.

Running Audacity using a touch interface on an Ultrabook 2 in 1 would transform the experience.

The Audacity / Intel Connection

Intel presented the idea of 2 in 1 awareness to a member of the Audacity Technical Leadership Council, Vaughan Johnson. Vaughan was so receptive to the idea that he used his 12 years of experience with Audacity to create the prototype used in the demos and images you see in this paper.

One of the most unique ways Audacity is being used today is as a recorder for ultrasonic frequencies; “sounds” that go well beyond the human ability to hear (20-20,000 Hz). A team studying porpoises is recording the sea mammal’s sound emissions accurately well above the human hearing spectrum. Another novel usage is a pyrotechnics engineer’s modification of the source code so he can control firework launches as his soundtrack plays through Audacity. Recordings made with Audacity produce a visual graph for human analysis—like hearing the sounds visually.

Vaughan believes over 200 current Audacity uses will need to be supported by the touch UI. Many of these uses make better sense when a tabletop is not available; which created a need for a more versatile touch-based interface. The power and performance of Intel®-based 2 in 1 computers helped Vaughan decide his first target platform for touch development. Tablets with less powerful processors don’t have the horsepower to run Audacity at a level that can support all those uses. The 2 in 1 technology provides Audacity with high quality audio on mobile devices without having to do a complete code rewrite, as would be required for other mobile OS platforms.

Adding Touch

The development team’s primary goal in modifying the user interface is simplification. Any screen element, be it a button, control, or status indicator, must pass scrutiny. Does the element support a simpler user experience? When a grandchild records her grandmother’s family stories, she is not an audio engineer, and the recording is best done at her grandmother’s convenience. So, what controls does a novice need on the screen to complete the capture successfully? In Vaughan’s own words, “Simplification—provide a basic recording device with ability to label important points / comments / crescendos / etc. by touch.”

Audacity needs a lot of performance that many tablets today can’t provide. On 64-bit Windows 8, the minimum hardware requirements to run Audacity are 2GB of RAM and a 1 GHz processor. Audacity recommends much higher if the application is to be used for recording lengthy multi-track projects.

Remove the Unnecessary

First thing is to remove the familiar Windows title bar. It has elements that aren’t necessary in the touch mode, such as application pane resizing. The menu bar is also not needed in the touch UI. The usual File, Edit, View, and Help buttons on the top of nearly every Windows traditional program are not present. The Audacity-specific control buttons Transport, Tracks, Generate, Effect, and Analyze are missing as well. Below these top two lines the Audacity laptop UI is very specialized. The area just above and below the audio spectrum display has no fewer than 43 visual control elements by default with room for adding more via customization.

Audio engineers love controls. Have you ever tried recording your 4-piece-band through a mixing board like the one below? Sound engineers prefer having an adjustment knob for every aspect of the soundscape.

For the person wanting to capture that special story from grandma, they prefer controls closer to the ones below.

The tablet UI has removed all but three controls from the top of the screen (microphone and speaker levels plus the Settings wrench) and placed only 20 remaining controls on the touch screen. Gone are numerical displays such as the scale of the spectrum display and the length of the selection on the bottom. They removed the information for file/encoding type, for audio hardware currently active, and the sampling rate of the project. These controls are still very relevant to the work Audacity does, however, they were just deemed less necessary when designing a simplified UI. These elements can still be accessed in the desktop UI.

Keep Only the Must-Haves

Audacity has grouped actions together that make sense, with a focus on ease-of-use. As you can see in the prototype, Audacity kept the audio spectrum graph largely unchanged, while altering the anchor/left side of each graph so it was touch friendly. A smaller number of channels are displayed in the simplified tablet interface. To add a track, the user only has to touch the + sign icon under current tracks, replacing the menu-bar nesting or hot-key combination Ctrl-Shift-N.

Icons, like a speaker instead of the mute button and headphone instead of the solo button, replace less efficient elements.

The most drastic changes in the touch UI are seen on the edges of the display. Where the laptop UI has a top to bottom flow, the touch UI has moved all the controls to the edge. In the prototype, transport controls (Record, Play, Pause, etc.) have been moved to the left and made larger for easy thumb access. Added to these are the Undo and Redo buttons. The last portion of the screen to be populated is the bottom. The remainder of the touch-enabled functions are at the bottom of the screen instead of the right-thumb favored right side to leave more screen real estate for left to right waveform display.

If more detailed control is required, users can manually transform to the laptop interface.

What’s behind the curtain?

To create the new UI, Vaughan Johnson primarily uses Microsoft Visual C 2008 on Windows. He also relies heavily on the body of development work the Audacity team built over the last 14 years including many cross-platform libraries. Vaughan is currently planning on porting the easy internal Windows standard controls as implemented by wxWidgets for touch UI. While the first of these descendant classes has been written, each subsequent layer is more complex and therefore needs to branch to simpler versions.

Summary

The task of adding a second UI to a popular, 14-year-old application like Audacity and its established user base will take time and effort. These features aren’t available for download today, but the work continues. If you would like to help or encourage the task, let your voice be heard at audacity.org. Better yet, join Vaughan, Roger, and many other developers at the Audacity Unconference in Preston England July 11^th and 12^th 2014!

Links:

Audacity Home Page
Audacity Wiki Page
Audacity Awards
Porpoise recording project
Firework control project
Audacity cross-platform libraries
Audacity Unconference
2 in 1 information page on Intel.com
Audacity 2-in-1 transition video demo

Additional information:

Detecting Slate/Clamshell Mode & Screen Orientation in a 2 in 1 Device by Seung-Woo Kim
Gameplay: Touch Controls and 2-in-1 Awareness for Your Favorite Games by Erik Niemeyer
Mixing Stylus and Touch Input on Windows* 8 by Meghana Rao
How to Write 2 in 1 Aware Applications by Stevan Rogers
Krita Gemini 2 in 1 UI Change demonstration Video on IDZ or YouTube*

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2014 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.