Google has two APIs / SDKs available in Developer Preview or
Sneak Peek status for working with Google Glass. The first is the Google Glass
Mirror API, which is a cloud based RESTful interface for sending "cards" to the
Google Glass timeline from remote servers. The second is the Glass Development
Kit, which builds onto the Android SDK for developing apps that run directly on
Glass. These apps can also take advantage of the Android NDK to incorporate
native code. Since it builds on the Android SDK, you don't need it for native
app development, but it does expose some nice additional features.
Appmethod and RAD Studio from Embarcadero Technologies support Android app development, including support for native Google Glass apps.
You can use your choice of C++
or Object Pascal programming languages. The support is based on both the
Android SDK and NDK. Once the Android ADB driver is loaded for Google Glass,
you can target it just like any other Android device. This article walks
through the ins and outs of Google Glass development with RAD Studio. The demos
in this article are in Object
Pascal and refer to RAD Studio, but everything you see here also works with
C++, or the individual C++Builder and Delphi products (with the Mobile Add-On
Pack). Likewise, most of this will work with Appmethod as well. You can download
a free 30-day license
for RAD Studio or Appmethod
to follow along with the steps in this article.
Google Glass Specifications
Google Glass is an Android platform device and so much more.
Essentially it is a miniaturized tablet you wear on your head. It has a
dual-core ARM Cortex-A9 CPU running ARM v7 NEON instructions, a camera, and
typical mobile device sensors. Thanks to the recent update it runs Android 4.4
KitKat (API 19). It is able to run many native Android apps, but that is just
the beginning.
The main output from glass is the 640x360 display, which looks
surprisingly good. Google claims it is the equivalent of a 25-inch screen from
8 feet away. It also has a bone conductive speaker and optional ear buds
for sound. It has a large array of input sensors though: a microphone,
accelerometer, gyroscope, magnetometer (compass), ambient light sensor,
proximity sensor and a forward facing wide-angle camera that can capture 5 mega
pixel photos or 720p video. Additionally, it has a touch pad on the right
temple, a sensor facing the eye for wink detection, and sensors for "On-Head
detection." Support for the latter two are still in beta.
Glass is not the first Android powered optical head-mounted
computer, but it may be the most innovative. Instead of a touch screen or
keyboard common on other Android devices, its primary input is voice and a
small touchpad on the right temple. The touch pad is not used to tap specific
widgets or apps, but typically provides four basic gestures: swiping forward and
backward for navigation, tapping for selecting, or swiping down for the back
button, which will exit your app if not otherwise handled.
The main user interface of Glass is the concept of a timeline
made up of a series of cards. One tap wakes up glass. Moving left or right from
moves you forward or backward through the timeline. From the home card, taping
or using a voice command brings up a list of installed apps which, when
launched, typically continue the left to right navigation paradigm.
Figure 1 - Google Glass Home Screen
Installing ADB USB Driver
Before you can develop for any Android device on Windows you need
to install the ADB USB Driver. This used to be much more complicated for Google
Glass. Now with revision 9 of the Google USB Driver, Google Glass has official
support. Simply run the Android Tools utility (installed with RAD
Studio) and it will bring up the Android SDK Manager. Select and install the
Google USB Driver found in the Extras section at the bottom of the Packages
list.
Figure 2 - Android SDK Manager
In device manager you will see an entry with a yellow triangle
under Other devices for Glass 1. You can load the driver from the
extras\google\usb_driver folder under your Android SDK install. Here is where
the folder is found if you installed the Android SDK with RAD Studio and used
the default settings:
C:\Users\Public\Documents\Embarcadero\Studio\14.0\PlatformSDKs\adt-bundle-windows-x86-20131030\sdk\extras\google\usb_driver
Creating Your First Native Glassware App
RAD Studio is great for creating true-native multi-device apps
that work across both Android and iOS devices. Because Glass is a specific
form-factor with a unique interface paradigm I will assume you are creating
this app just for Glass. If you do want to make this app work on multiple
devices though, you certainly can by using typical multi-device techniques.
RAD Studio provides a number of different templates to get you
started creating an application. We will start with just a Blank Application,
which will probably be the most common starting point for most Glassware,
although the 3D Application could also be really useful.
Figure 3 - FireMonkey Mobile Application Wizard
Setting Up the IDE for Google Glass
While the actual resolution of the main display is 640x360, its
pixel density is 1.5. This means an effective resolution for control placement
of 472x240 (divide the actual pixels by the pixel density). RAD Studio includes
a device layout for Google Glass already sized correctly. You can select it
from the drop down box above the design surface in the IDE.
Figure 4 - Design Surface with Google Glass Layout
This layout will work with any color of Google Glass. Likewise,
you could define your own Google Glass device in the Device Manager (Tools
-> Options -> Environment Options -> Form Designer -> Device
Manager). Add a new device for the Android platform. You only need to enable
the Landscape Left orientation. You will need to provide a background image
that is at least 472x240 pixels. You can specify a Status bar size if you want,
but you will typically be hiding it on a Glass app (more about that in a bit).
Voice Launch Support
When you run your app it will launch automatically, but once it
closes you will not find it on the menu again. This is because Glass uses a
different launch mechanism than a traditional Android app. It is easy enough to
add support though.
Once you've built your app for the first time, RAD Studio
generates an AndroidManifest.template.xml file. This is a template used to
generate the actual AndroidManifest.xml for your Android app.
I find it easiest to add this file to the project group to make
it easy to open and edit as needed. Just right click on your project file and
select Add . . . This brings up the file browse dialog so you can select
the AndroidMainfest.template.xml.
Now you can edit the template by double clicking on it from the Project
Manager.
There are two edits you need to make in your manifest. The first
is inside the <intent-filter> element. You need to specify that you want
to respond to the Voice Trigger action. Do this by adding the element:
<action android:name="com.google.android.glass.action.VOICE_TRIGGER" />
Then you need to add meta data for this action inside the
<activity>
element:
<meta-data android:name="com.google.android.glass.VoiceTrigger" android:resource="@xml/voice_trigger_start" />
This specifies that the XML file voice_trigger_start.xml will
define your voice trigger. This file can have any name, but when you add the
file it needs to match the name specified here. It should be in the XML
directory though.
One note about selecting a voice trigger: Google has a list of approved
voice triggers, but if you want to use a different one you are only allowed
to do so during development and by adding a special permission to your
manifest. Immediately after the <%uses-permission%> add the following:
<uses-permission android:name="com.google.android.glass.permission.DEVELOPMENT" />
You can submit your suggested voice trigger to Google and if
approved they will add it to the list.
Now you need to create a voice_trigger_start.xml file.
- Add a new XML File to your project by right-clicking the Project Manager and selecting Add New > Other > Web Documents > XML File.
- Rename the XML File to voice_trigger_start.xml or whatever name you specified in the manifest. The manifest does not include the .xml extension.
- Save the XML File to your project's folder.
- Edit the XML File to look like the following:
="1.0" ="UTF-8"
<trigger keyword="Your keyword here" />
Replace Your
keyword here with whatever
phrase or keyword you want to use if you have the Development permission
specified. Now when your app is deployed to Glass then it will show up in the
menu system with the app icon.
Lastly, you need to configure the deployment of your
voice_trigger_start.xml file to be deployed in the correct location of your
Android app’s apk package. This is done through the deployment manager which is
accessible via the Project > Deployment menu option. Simply click the
Add files button (second from the left) and browse to and select the voice_trigger_start.xml
file you created. After you add the file you will need to edit the Remote
Path to res/xml, which will place it in the correct location within
the apk bundle.
Common Glass App Configuration
Commonly Android Apps have a status
bar at the top of the screen. On Google Glass this status bar is hidden. Hiding
it for your app is a two-step process. First you can hide it at design time by
setting the BoarderStyle of the form to None. To remove the
status bar at runtime, go to Project -> Options -> Version Info and
select All configurations – Android Platform at the top. Then change the
value for the theme key from TitleBar to No TitleBar.
Figure 5 - Project Options - Theme No TitleBar
The default Android theme is Halo Light, which is black
and grey on white background. For a phone or tablet that is fine, but on glass
white pixels are very bright, while black pixels are basically transparent. A
mostly white screen that close to your eye is akin to having a flashlight
shining in your eye, which isn’t very comfortable. Thankfully RAD Studio includes
a Halo Dark, and a specialized Google Glass style. The Google Glass style is
black with light colorful text and a slightly larger font size. It is based on
the recommended Google Glass style.
To change the style of your app, add a TStyleBook to your form
from the Tool Palette. Make sure Android is selected as the Target
Platform in the Project Manager. Select the TForm for your application
(the blank design surface) and in the Object Inspector set the StyleBook
property to the TStyleBook you just placed on your form. Now double click on
the TStyleBook and then click the Load button and navigate to the styles folder.
By default is found in:
C:\Users\Public\Documents\Embarcadero\Studio\14.0\Styles\Android
Glass apps also typically have a silhouetted white app
icon on a transparent background. You can change the launcher icon for your app
from the Application node of the Project Options dialog.
Building Your App
At this point you are ready to build an app for Google
Glass that will integrate into the Glass environment. Since Glass is a
different usage paradigm than typical mobile apps you will need to put some
thought into your app design. The following are some tips on how to interact
with different elements of Glass for the user experience.
Using the Sensors
Glass lacks a keyboard or touch screen, but still has all
the standard sensors of a mobile device. You can use the standard sensor
components to access these sensors.
Location and GPS
Glass has a built in GPS. The TLocationSensor
has
an OnLocationChanged
event that occurs when the component is active, and
the location changes by more than the Distance property. Inside the OnLocationChanged
event handler there is a NewLocation parameter that contains the
latitude and longitude of the new location.
Motion & Orientation Tracking
To track the movement and direction of the Glass and the
wearer’s head use the TMotionSensor and TOrientationSensor
component. They offer information about acceleration, angle, state, heading,
speed and motion of the device. Since the Glass is worn on the users head this
information is translated into head movement and where the user is looking and
how they are moving their head.
It actually accesses information from a number of
different sensors, including the gyroscope, magnetometer and accelerometer. You
can use a TTimer to poll the sensor information to discover information
about the motion and orientation over time, or just read the data once for one
time information.
Accessing the Camera
The TCameraComponent gives you the ability to grab
frames from the camera. You can grab multiple frames a second to provide a
preview or just grab an individual frame. Google requires that the display be
on when you are capturing a picture, but doesn’t require that the display show
the image you are grabbing (although that is more convenient for your user).
To use the TCameraComponent, set the Active
property to true, and then respond to the OnSampleBufferReady
event. Keep
in mind that this event is not occurring in the UI thread, so if you want to
update the UI to display the image to the user you will want to use code
similar to the following:
procedure TMainForm.CameraComponent1SampleBufferReady(
Sender: TObject; const ATime: Int64);
Begin
TThread.Synchronize(TThread.CurrentThread, GetImage);
end;
procedure TMainForm.GetImage;
begin
CameraComponent1.SampleBufferToBitmap(imgCameraView.Bitmap, True);
end;
Using the Touchpad
The easiest way to interact with the touchpad is through
interactive gestures. The Pan interactive gesture gives you series of events
when the user brushes their finger on the touch pad. For example, to allow the
user to navigate between options in your app you could look for horizontal
movement of the pan gesture.
To respond to interactive gestures, simply expand the Touch
property of the form and then specify the individual interactive gestures you
want to receive events for. After that, the OnGesture
event will occur
when the user makes an interactive gesture. Here is an example gesture handler
that allows the user to set the screen timeout based on horizontal position of
the interactive gesture.
procedure TMainForm.FormGesture(Sender: TObject;
const EventInfo: TGestureEventInfo; var Handled: Boolean);
var
x: Integer;
begin
if EventInfo.GestureID = igiPan then
begin
if TInteractiveGestureFlag.gfBegin in EventInfo.Flags then
begin
fPanStart := EventInfo.Location;
fStartVal := pbTimeOut.Value;
end
else
if TInteractiveGestureFlag.gfEnd in Eventinfo.Flags then
begin
SetScreenOffTimeout(Trunc(pbTimeOut.Value * 1000));
end
else
begin
if not (TInteractiveGestureFlag.gfInertia in EventInfo.Flags) then
begin
x := Trunc((fPanStart.X - EventInfo.Location.X)/10) * 10;
pbTimeOut.Value := fStartVal + x;
end;
DisplayTimeOut(Trunc(pbTimeOut.Value));
end;
end;
end;
One note, if the user swipes down on the touch pad then the
back button event is triggered in addition to the pan gesture. You can handle
this back button yourself by trapping it. Simply add an OnKeyUp
event
handler to your form with the following code:
if Key = vkHardwareBack then
begin
Key := 0;
end;
If you always set Key to 0 then users will not be able to exit
your app, so plan appropriately.
Conclusion
RAD
Studio and Appmethod’s support for
Android in general and Google Glass specifically make it a great choice for
developing native Glassware. With support for C++ and Object Pascal, you can
use the language you prefer. RAD Studio ships with a native
Glassware sample app that demonstrates a lot of what is mentioned here. The
documentation also includes a First
Steps with Google Glass guide with details of the development process.
There are other examples
& snippets that demonstrate the use of the different sensors, gestures,
etc. It is also possible to call into the GDK to access the timeline or other
Glass specific features. The GDK is still under constant updates. Look for future
articles covering how to use the GDK on my blog at
Delphi.org.