Developing Native Google Glass Apps

Google has two APIs / SDKs available in Developer Preview or Sneak Peek status for working with Google Glass. The first is the Google Glass Mirror API, which is a cloud based RESTful interface for sending "cards" to the Google Glass timeline from remote servers. The second is the Glass Development Kit, which builds onto the Android SDK for developing apps that run directly on Glass. These apps can also take advantage of the Android NDK to incorporate native code. Since it builds on the Android SDK, you don't need it for native app development, but it does expose some nice additional features.

Appmethod and RAD Studio from Embarcadero Technologies support Android app development, including support for native Google Glass apps. You can use your choice of C++ or Object Pascal programming languages. The support is based on both the Android SDK and NDK. Once the Android ADB driver is loaded for Google Glass, you can target it just like any other Android device. This article walks through the ins and outs of Google Glass development with RAD Studio. The demos in this article are in Object Pascal and refer to RAD Studio, but everything you see here also works with C++, or the individual C++Builder and Delphi products (with the Mobile Add-On Pack). Likewise, most of this will work with Appmethod as well. You can download a free 30-day license for RAD Studio or Appmethod to follow along with the steps in this article.

Google Glass Specifications

Google Glass is an Android platform device and so much more. Essentially it is a miniaturized tablet you wear on your head. It has a dual-core ARM Cortex-A9 CPU running ARM v7 NEON instructions, a camera, and typical mobile device sensors. Thanks to the recent update it runs Android 4.4 KitKat (API 19). It is able to run many native Android apps, but that is just the beginning.

The main output from glass is the 640x360 display, which looks surprisingly good. Google claims it is the equivalent of a 25-inch screen from 8 feet away. It also has a bone conductive speaker and optional ear buds for sound. It has a large array of input sensors though: a microphone, accelerometer, gyroscope, magnetometer (compass), ambient light sensor, proximity sensor and a forward facing wide-angle camera that can capture 5 mega pixel photos or 720p video. Additionally, it has a touch pad on the right temple, a sensor facing the eye for wink detection, and sensors for "On-Head detection." Support for the latter two are still in beta.

Glass is not the first Android powered optical head-mounted computer, but it may be the most innovative. Instead of a touch screen or keyboard common on other Android devices, its primary input is voice and a small touchpad on the right temple. The touch pad is not used to tap specific widgets or apps, but typically provides four basic gestures: swiping forward and backward for navigation, tapping for selecting, or swiping down for the back button, which will exit your app if not otherwise handled.

The main user interface of Glass is the concept of a timeline made up of a series of cards. One tap wakes up glass. Moving left or right from moves you forward or backward through the timeline. From the home card, taping or using a voice command brings up a list of installed apps which, when launched, typically continue the left to right navigation paradigm.

Installing ADB USB Driver

Before you can develop for any Android device on Windows you need to install the ADB USB Driver. This used to be much more complicated for Google Glass. Now with revision 9 of the Google USB Driver, Google Glass has official support. Simply run the Android Tools utility (installed with RAD Studio) and it will bring up the Android SDK Manager. Select and install the Google USB Driver found in the Extras section at the bottom of the Packages list.

In device manager you will see an entry with a yellow triangle under Other devices for Glass 1. You can load the driver from the extras\google\usb_driver folder under your Android SDK install. Here is where the folder is found if you installed the Android SDK with RAD Studio and used the default settings:

C:\Users\Public\Documents\Embarcadero\Studio\14.0\PlatformSDKs\adt-bundle-windows-x86-20131030\sdk\extras\google\usb_driver

Creating Your First Native Glassware App

RAD Studio is great for creating true-native multi-device apps that work across both Android and iOS devices. Because Glass is a specific form-factor with a unique interface paradigm I will assume you are creating this app just for Glass. If you do want to make this app work on multiple devices though, you certainly can by using typical multi-device techniques.

RAD Studio provides a number of different templates to get you started creating an application. We will start with just a Blank Application, which will probably be the most common starting point for most Glassware, although the 3D Application could also be really useful.

Setting Up the IDE for Google Glass

While the actual resolution of the main display is 640x360, its pixel density is 1.5. This means an effective resolution for control placement of 472x240 (divide the actual pixels by the pixel density). RAD Studio includes a device layout for Google Glass already sized correctly. You can select it from the drop down box above the design surface in the IDE.

This layout will work with any color of Google Glass. Likewise, you could define your own Google Glass device in the Device Manager (Tools -> Options -> Environment Options -> Form Designer -> Device Manager). Add a new device for the Android platform. You only need to enable the Landscape Left orientation. You will need to provide a background image that is at least 472x240 pixels. You can specify a Status bar size if you want, but you will typically be hiding it on a Glass app (more about that in a bit).

Voice Launch Support

When you run your app it will launch automatically, but once it closes you will not find it on the menu again. This is because Glass uses a different launch mechanism than a traditional Android app. It is easy enough to add support though.

Once you've built your app for the first time, RAD Studio generates an AndroidManifest.template.xml file. This is a template used to generate the actual AndroidManifest.xml for your Android app.

I find it easiest to add this file to the project group to make it easy to open and edit as needed. Just right click on your project file and select Add . . . This brings up the file browse dialog so you can select the AndroidMainfest.template.xml. Now you can edit the template by double clicking on it from the Project Manager.

There are two edits you need to make in your manifest. The first is inside the <intent-filter> element. You need to specify that you want to respond to the Voice Trigger action. Do this by adding the element:

<action android:name="com.google.android.glass.action.VOICE_TRIGGER" />

Then you need to add meta data for this action inside the <activity> element:

<meta-data android:name="com.google.android.glass.VoiceTrigger" android:resource="@xml/voice_trigger_start" />

This specifies that the XML file voice_trigger_start.xml will define your voice trigger. This file can have any name, but when you add the file it needs to match the name specified here. It should be in the XML directory though.

One note about selecting a voice trigger: Google has a list of approved voice triggers, but if you want to use a different one you are only allowed to do so during development and by adding a special permission to your manifest. Immediately after the <%uses-permission%> add the following:

<uses-permission android:name="com.google.android.glass.permission.DEVELOPMENT" />

You can submit your suggested voice trigger to Google and if approved they will add it to the list.

Now you need to create a voice_trigger_start.xml file.

1. Add a new XML File to your project by right-clicking the Project Manager and selecting Add New > Other > Web Documents > XML File.
2. Rename the XML File to voice_trigger_start.xml or whatever name you specified in the manifest. The manifest does not include the .xml extension.
3. Save the XML File to your project's folder.
4. Edit the XML File to look like the following:

<?xml version="1.0" encoding="UTF-8"?>

<trigger keyword="Your keyword here" />

Replace Your keyword here with whatever phrase or keyword you want to use if you have the Development permission specified. Now when your app is deployed to Glass then it will show up in the menu system with the app icon.

Lastly, you need to configure the deployment of your voice_trigger_start.xml file to be deployed in the correct location of your Android app’s apk package. This is done through the deployment manager which is accessible via the Project > Deployment menu option. Simply click the Add files button (second from the left) and browse to and select the voice_trigger_start.xml file you created. After you add the file you will need to edit the Remote Path to res/xml, which will place it in the correct location within the apk bundle.

Common Glass App Configuration

Commonly Android Apps have a status bar at the top of the screen. On Google Glass this status bar is hidden. Hiding it for your app is a two-step process. First you can hide it at design time by setting the BoarderStyle of the form to None. To remove the status bar at runtime, go to Project -> Options -> Version Info and select All configurations – Android Platform at the top. Then change the value for the theme key from TitleBar to No TitleBar.

The default Android theme is Halo Light, which is black and grey on white background. For a phone or tablet that is fine, but on glass white pixels are very bright, while black pixels are basically transparent. A mostly white screen that close to your eye is akin to having a flashlight shining in your eye, which isn’t very comfortable. Thankfully RAD Studio includes a Halo Dark, and a specialized Google Glass style. The Google Glass style is black with light colorful text and a slightly larger font size. It is based on the recommended Google Glass style.

To change the style of your app, add a TStyleBook to your form from the Tool Palette. Make sure Android is selected as the Target Platform in the Project Manager. Select the TForm for your application (the blank design surface) and in the Object Inspector set the StyleBook property to the TStyleBook you just placed on your form. Now double click on the TStyleBook and then click the Load button and navigate to the styles folder. By default is found in:

C:\Users\Public\Documents\Embarcadero\Studio\14.0\Styles\Android

Glass apps also typically have a silhouetted white app icon on a transparent background. You can change the launcher icon for your app from the Application node of the Project Options dialog.

Building Your App

At this point you are ready to build an app for Google Glass that will integrate into the Glass environment. Since Glass is a different usage paradigm than typical mobile apps you will need to put some thought into your app design. The following are some tips on how to interact with different elements of Glass for the user experience.

Using the Sensors

Glass lacks a keyboard or touch screen, but still has all the standard sensors of a mobile device. You can use the standard sensor components to access these sensors.

Location and GPS

Glass has a built in GPS. The TLocationSensor has an OnLocationChanged event that occurs when the component is active, and the location changes by more than the Distance property. Inside the OnLocationChanged event handler there is a NewLocation parameter that contains the latitude and longitude of the new location.

Motion & Orientation Tracking

To track the movement and direction of the Glass and the wearer’s head use the TMotionSensor and TOrientationSensor component. They offer information about acceleration, angle, state, heading, speed and motion of the device. Since the Glass is worn on the users head this information is translated into head movement and where the user is looking and how they are moving their head.

It actually accesses information from a number of different sensors, including the gyroscope, magnetometer and accelerometer. You can use a TTimer to poll the sensor information to discover information about the motion and orientation over time, or just read the data once for one time information.

Accessing the Camera

The TCameraComponent gives you the ability to grab frames from the camera. You can grab multiple frames a second to provide a preview or just grab an individual frame. Google requires that the display be on when you are capturing a picture, but doesn’t require that the display show the image you are grabbing (although that is more convenient for your user).

To use the TCameraComponent, set the Active property to true, and then respond to the OnSampleBufferReady event. Keep in mind that this event is not occurring in the UI thread, so if you want to update the UI to display the image to the user you will want to use code similar to the following:

// Event handler for the CameraComponent’s SampleBufferReady event
procedure TMainForm.CameraComponent1SampleBufferReady(
  Sender: TObject; const ATime: Int64);
Begin
  // Use Synchronize to move the execution to the main UI thread.
  TThread.Synchronize(TThread.CurrentThread, GetImage);
end;
 
procedure TMainForm.GetImage;
begin
  // imgCameraView is a TImage component on the UI for displaying the image
  CameraComponent1.SampleBufferToBitmap(imgCameraView.Bitmap, True);
end;

Using the Touchpad

The easiest way to interact with the touchpad is through interactive gestures. The Pan interactive gesture gives you series of events when the user brushes their finger on the touch pad. For example, to allow the user to navigate between options in your app you could look for horizontal movement of the pan gesture.

To respond to interactive gestures, simply expand the Touch property of the form and then specify the individual interactive gestures you want to receive events for. After that, the OnGesture event will occur when the user makes an interactive gesture. Here is an example gesture handler that allows the user to set the screen timeout based on horizontal position of the interactive gesture.

procedure TMainForm.FormGesture(Sender: TObject;
  const EventInfo: TGestureEventInfo; var Handled: Boolean);
var
  x: Integer;
begin
  // Handle the Pan Interactive Gesture
  if EventInfo.GestureID = igiPan then
  begin
    // Specific processing for the beginning of the gesture – save the start location
    if TInteractiveGestureFlag.gfBegin in EventInfo.Flags then
    begin
      fPanStart := EventInfo.Location;
      fStartVal := pbTimeOut.Value;
    end
    else
    // Specific processing at the end of the gesture – set the timeout
    if TInteractiveGestureFlag.gfEnd in Eventinfo.Flags then
    begin
      SetScreenOffTimeout(Trunc(pbTimeOut.Value * 1000));
    end
    else
    // Update the display based on the current gesture position
    begin
      if not (TInteractiveGestureFlag.gfInertia in EventInfo.Flags) then
      begin
        x := Trunc((fPanStart.X - EventInfo.Location.X)/10) * 10;
        pbTimeOut.Value := fStartVal + x;
      end;
      DisplayTimeOut(Trunc(pbTimeOut.Value));
    end;
  end;
end;

One note, if the user swipes down on the touch pad then the back button event is triggered in addition to the pan gesture. You can handle this back button yourself by trapping it. Simply add an OnKeyUp event handler to your form with the following code:

if Key = vkHardwareBack then
begin  
  // Do whatever you want to do here  
  Key := 0;
  // Set Key = 0 if you want to prevent the default action
end;

If you always set Key to 0 then users will not be able to exit your app, so plan appropriately.

Conclusion

RAD Studio and Appmethod’s support for Android in general and Google Glass specifically make it a great choice for developing native Glassware. With support for C++ and Object Pascal, you can use the language you prefer. RAD Studio ships with a native Glassware sample app that demonstrates a lot of what is mentioned here. The documentation also includes a First Steps with Google Glass guide with details of the development process. There are other examples & snippets that demonstrate the use of the different sensors, gestures, etc. It is also possible to call into the GDK to access the timeline or other Glass specific features. The GDK is still under constant updates. Look for future articles covering how to use the GDK on my blog at Delphi.org.