HoloLens first impression

Introduction

With big probability I'm going to participate in new project that is related to Microsoft HoloLens. Recently I received HoloLens device to play with it and I'm writing this article and probably few more to document my first steps with this technology and device. I want to share with you my first impressions and the issues I solved when doing first steps with this exciting thing. At the moment I'm keeping HoloLens device in my hand and exciting to start this journey and dive into HoloLens world.

DEAR READERS I WILL DESCRIBE HERE SOME THINGS THAT MIGHT BE TRIVIAL AND SIMPLE. PLEASE BE PATIENT TO THIS AS I TARGET THIS ARTICLE NOT ONLY FOR PEOPLE THAT ARE PRACTICING WITH HOLOLENS BUT ALSO FOR THOSE THAT WANT TO SEE WHAT IS IT WITHOUT PREPARING ENVIRONMENT AND DO HANDS ON THIS.

About terminology and idea

HoloLens project idea is to create holograms that are built by light and sound and that are placed in the environment where you're now using HoloLens. The idea is to combine real objects and holograms so they both will provide you with new mixed reality where you should feel holographic objects quite similar as real and even more attractive as they are software applications and can bring more input that real object without algorithms inside. In HoloLens you can interact with holograms using your:

•  Gaze - HoloLens use your gaze to understand where you're looking now. By analysis of gaze you may understand where user is focused now and if his gaze is related to your HoloLens application. HoloLens doesn't determine your gaze from your eyes it uses position of your head to create a vector of your gaze. Microsoft recommend to imagine this vector as a laser ray. When you move your head HoloLens uses this ray to determine real and holographic objects you're looking now and tries to interact with them if required. HoloLens uses cursor to indicate the direction of the gaze, this is small circle pointer usually. You can change the visual style of the gaze pointer but it is recommended to use it in your HoloLens application. Once you found real or holographic object by your gaze you can interact with it. This can be done by gestures or voice.

Picture below shows the gaze pointer:

• Gesture - as mentioned above, once you've targeted object with gaze you can interact with it using gesture. In HoloLens device there are several cameras that scan and recognize your movemens and hand's gesrues. It recognizes hands gestures in front of your device. Also it has clicker device that emulate gesture. HoloLens recognize input from any of your hands. HoloLens scan your hands all the time but it recognizes only three gestures: ready gesture (back of your hadn is looking at you and index finger is up), press gesture (same as previous one but index finger is down),  bloom gesture (To do the bloom gesture, hold out your hand, palm up, with your fingertips together. Then open your hand). All the other positions of your hands are ignored. Bloom gesture is used for going to home screen and is utilized by system, you can't use it in your application.

By combining ready and pressed gestures with hand movements HoloLens gives ability to create more complex gesture inputs such as:

• Hold - this is something similar to touch and hold in touch screen or mouse down event when you use mouse input
• Manipulation - used to play with your hologram. You can move it, rotate or resize. Manipulation starts with the press gesture.
• Navigation - used to navigate thru stuff by hand. You use the press gesture to start navigation and then move your hand to navigate.

• Voice commands - first and one of the most important voice commands in device is "select". It is the same as press and release gesture or clicker click. You move your gaze to some object and say "select", the result is similar to as you did it with gestures or clicker. Cortana is built in HoloLens device this assistance usage is very convenient in a device like HoloLens. There is list of commands that Cortana may use specifically on HoloLens device, in parallel you can use to do web search on other voice input from you. "Hey Cortana" command will call assistant from any place and at any time of your HoloLens usage.

Experience and interesting findings when using device

Below is list of things that were interesting, unusual, surprised or disappointed me:

• You can call Cortana at any moment of time by telling "Hey, Cortana". This is convenient. Cortana appears on top of some applications, this is convenient as you can ask it to make a picture of that application for example.
•  Don't try this device when you sit or lay only. The beauty of holograms and device best experience is when you move around holograms and not only sit or lay. When I first used device I put all applications at white wall so I had a feeling that this is pretty same laptop\desktop experience but without keyboard and mouse and with helmet in your head. The truth is different HoloLens is amazing and completely different experience and you feel it 100% only if you move with device and not use holograms as windows on your walls
• Home screen is called by bloom gesture and is accessible at any point of time as well as Cortana. Also very convenient.
• It is not clear how to close some apps as they don't have any frame and everything you can do is call Cortana or switch to start screen.
• I couldn't make screenshot of home screen as when you call Cortana from it, Cortana executes its own UI from there. Hope I'll find how to do this in the future
• Many current applications use 2d view of them and there is no big difference between using them in HoloLens or your laptop\desktop. True HoloLens stuff are holograms, they are really cool. Current limitation (it seems to be for future as well) is that you can run only one app that renders hologram\s at the specific moment of time.

2D view:

3D view:

• Speech recognition isn't perfect. I'm not native speaker, but can pronounce simple command as "Remove" pretty clearly. Quite often when I looked at menu with highlighted text command and pronounced it, no speech recognition happened and I had to click.
• Kids are happy with this device, they are people of 21st century and it seems like they were born to use such stuff. My 4 years old son can spend hours by looking and playing with holograms or games. Especially holograms from Actiongram app from Microsoft

 

Prerequisites and development environment configuration

Before you start working with HoloLens you need to have proper environment configured. There is a description of required things by the following link: https://developer.microsoft.com/en-us/windows/holographic/install_the_tools. I think Microsoft updates it frequently so before you will start something go and check that you have everything required for this work. At the moment of this article writing I have Windows 10 Enterprise, Visual Studio 2015 Enterprise with Update 3 and Windows SDK of version 10 version 10.0.14393.0.

Next step you need to go to Control Panel -> Programs -> Programs and Features -> Turn Windows Features on or off -> ensure that "Hyper-V" is selected to perform following installations:

• install HoloLens Emulator. You can get it by following link. Also here you can read about emulator usage.
• Install Unity Engine version 5.5
• Install Vuforia version 6.2 (I really don't know what is it at the moment and do this installation according to Microsoft recommendations)

If you did everything above you will have following type of project in your Visual Studio:

 

Creating default app and run it with simulator

I've created my first application and named it HoloTest. Just after I clicked OK to create a solution Visual Studio checked with me about target versions of Windows for my app. I want to say you that default values were pretty fresh Windows 10 releases for the moment of this text writing:

I used default values and agreed with what VS proposed for me. After pressing OK I have my first project for HoloLens created and default files are generated by Visual Studio for it. Let's review what we have here, but before doing it I want to build and try to run default project first to see if Microsoft prepared something simple that you can run to have a feeling infrastructure and environment for HoloLens. According to Microsoft for this purpose we need to have selected:

• Configuration: Debug
• Platform: x86
• HoloLens Emulator 10.0.14393.0

 

Note: before starting emulator successfully I experienced two issues:

• Selected not HoloLens Emulator but Simulator it didn't start. It seems obvious, but I Googled for solution :)
• Changed my Dell Ultrabook to another laptop as Dell had 4GB of RAM and all my attempts to run emulator on this hardware failed. 8GB of RAM is requirement here and I wasn't able to solve it by different tricks as changing registry, changing system requirements for emulator in HyperV, increasing swap file size etc.

Once everything is done I can see default project executed in my Emulator. This is nice multicolor cube that you can move and that is scrolled all the time. This is how it looks at the screen:

You can manipulate by cube in the screen by moving around it and emulate your head and body using keyboard.

In emulator tools there is simulation tab where you can see coordinates values when you emulate your body moving.

Below is list of things I tried to control:

• Yaw: turning left or right, controlled by pressing left or right arrow
• Pitch: turn up or down, controlled by pressing up or down arrow
• Roll: controlled by Q or E
• X coordinate: controlled by A and D
• Y coordinate: controlled by Page Up and Page Down
• Z coordinate: controlled by W and S

Below is sample of what I achieved with my cube:

Below are new values of changed coordinates:

More details about emulator controlling you can find here.

Deploying to device and execute there

Deploying to device from VS is pretty simple and all you need is to have HoloLens device and computer with Visual Studio connected together using USB cable. Below are steps you need to perform:

 

1. In VS select deploy to device:

2. In HoloLens turn on Developers Mode:

3. Pair computer and HoloLens device:

!!!Note: HoloLens generates new PIN code all the time and you need to keep dialog with it opened until you enter this PIN in Visual Studio.  Once you inserted everything correctly, deployment will finish and you will be able to run your app in device.

4. Voilà, we have our cool colored cube in device:

Conclusions

HoloLens device and augmented reality idea in Microsoft realization is breakthrough. Even so technology is only on its beginning and there are not so much devices available you can feel that Microsoft did a great job and your feeling about device is pretty cool. I'm exciting to start working with this device and technology and hope to write few more technical articles where I'll show you my real or demo applications and not only default project from Visual Studio.