Day 71 of 100 Days of VR: How To Create A Menu That Stays In Your Screen In Unity

Introduction

Usually, when we make a menu in Unity, we would just have it show up on our screen. However, in VR, there’s no “screen”, we see everything in the first person and it would be uncomfortable to have the menu stay in front of our face until we close it.

We could leave the menu at where we created it, but let’s say it’s an important menu that we want the user to focus on until they close it.

What are we going to do to solve this problem?

We’ll create a menu that appears in front of the player like normal, however, to make this more “VR friendly” is that we’ll allow the player to look around the screen, however, the moment the menu leaves the screen, we’ll start moving the screen to be in front of the player.

How are we going to implement this?

A lot of the required code is something we already know, like moving a game object to be in front of our player.

The only part we haven’t learned how to do is to detect if a UI element is visible. We could easily check if a game object is visible to a camera with a renderer component, unfortunately for a UI element, we don’t have a renderer component.

Instead, I found a great solution provided for us by KGC in this thread: Test if UI element is visible on the screen. Thanks for the awesome solution!

Now that we know what we want to build and have some idea on how to build it, let’s get started!

Creating the Following Menu

Now at this point, we have an idea of what we want to make. Let’s go make it. To do that, we’re going to make a new script that will help us deal with the state.

We’re going to call this class ViewHelper.

In MenuContainer game object, we’re going to create a new script. We’re going to call it ViewHelper.

In ViewHelper, we’re going to do some very similar state management that we did with MenuController, but it’s not going to be that complicated.

Here’s what ViewHelper looks like:

using System.Collections;
using UnityEngine;

public class ViewHelper : MonoBehaviour
{    
    private RectTransform _canvasRectTransform; // holds the rectangle transform of our menu
    private Camera _mainCamera;
    private float _yGap; // Keeps our menu up so the center is in the front of our gaze.

    enum State { InScreen, NotInScreen, Moving }; // The state our game object can be in
    private State _currentState;
    
    void Start ()
    {
        _canvasRectTransform = GetComponentInChildren<RectTransform>();
        _mainCamera = Camera.main;
        _currentState = State.InScreen;
        _yGap = 2f;
    }
    
    void Update ()
    {
        switch (_currentState)
        {
            case State.InScreen:
                if (!IsFullyVisibleFrom(_canvasRectTransform, _mainCamera))
                {
                    // If the menu isn't fully visible anymore switch to NotInScreen state.
                    _currentState = State.NotInScreen;
                }
                break;
            case State.NotInScreen:
                // If the menu isn't in the screen anymore, start moving it towards the player.
                _currentState = State.Moving;
                StartCoroutine(MoveToFrontOfPlayer());
                break;
        }
    }

    // Get the Vector location of the forward of our camera with some 
    // distance adjustments.
    private Vector3 GetCameraFoward()
    {
        Vector3 forward = _mainCamera.transform.forward * 3;
        forward.y += _yGap;
        return forward;
    }

    // Coroutine that will move our menu to the front of our player every frame
    private IEnumerator MoveToFrontOfPlayer()
    {
        // While we're not directly in front of the player, 
        // slowly move the menu to the front of our player
        while (transform.position != GetCameraFoward())
        {
            transform.eulerAngles = _mainCamera.transform.eulerAngles; // set our angle to be 
                                                      // the same as the one the camera is facing
            float speed = 4f * Time.deltaTime;        // the speed we're going to move the position 
                                                      // of our camera
            transform.position = Vector3.MoveTowards
                      (transform.position, GetCameraFoward(), speed); // move the position of our
                                                                      // menu to our camera's forward.
            yield return null;
        }
        _currentState = State.InScreen; // change back to our normal state 
                                        // after the menu goes back to the front of the camera
    }

    /// <summary>
    /// Counts the bounding box corners of the given RectTransform 
    /// that are visible from the given Camera in screen space.
    /// </summary>
    /// <returns>The amount of bounding box corners that are visible 
    /// from the Camera or -1 if a corner isn't in the screen</returns>
    /// <param name="rectTransform">Rect transform.</param>
    /// <param name="camera">Camera.</param>
    private int CountCornersVisibleFrom(RectTransform rectTransform, Camera camera)
    {
        Rect screenBounds = new Rect(0f, 0f, Screen.width, Screen.height); // Screen space 
                                  // bounds (assumes camera renders across the entire screen)
        Vector3[] objectCorners = new Vector3[4];
        rectTransform.GetWorldCorners(objectCorners);

        int visibleCorners = 0;
        for (var i = 0; i < objectCorners.Length; i++) // For each corner in rectTransform
        {
            Vector3 tempScreenSpaceCorner = 
                    camera.WorldToScreenPoint(objectCorners[i]); // Transform world space 
                                                         // position of corner to screen space
            if (screenBounds.Contains(tempScreenSpaceCorner)) // If the corner is inside the screen
            {
                visibleCorners++;
            }
            else
            {
                return -1;
            }
        }
        return visibleCorners;
    }

    /// <summary>
    /// Determines if this RectTransform is fully visible from the specified camera.
    /// Works by checking if each bounding box corner of this RectTransform 
    /// is inside the cameras screen space view frustrum.
    /// </summary>


    /// <returns><c>true</c> if is fully visible from the specified camera; 
    /// otherwise, <c>false</c>.</returns>
    /// <param name="rectTransform">Rect transform.</param>
    /// <param name="camera">Camera.</param>
    private bool IsFullyVisibleFrom(RectTransform rectTransform, Camera camera)
    {
        return CountCornersVisibleFrom(rectTransform, camera) == 4; // True if all 
                                                                    // 4 corners are visible
    }
}

Looking at the Variables

Here’s what we’re going to be working with:

• private RectTransform _canvasRectTransform – The RectTransform of our menu which we will use to see if it’s on the screen or not.
• private Camera _mainCamera – Shortcut to refer to our main camera.
• private float _yGap – A value that I used to keep the center of the menu to be at the front of our gaze. Specifically, wherever we’re looking at + the y value the menu was at when we first started.
• private State _currentState – The state that we are in. There are 3 states our menu can be in: on the screen, not in the screen, and then moving back to the center of the screen.

Walking Through the Code

Now that we have a sneak peek of what we’re doing with our fields, let’s look and see how the code works.

1. Once again, we initialize everything in Start().
2. In Update(), we’re once again running different lines of code based off of what state we’re currently in. If we’re currently in the InScreen state, we’ll have to constantly check if we’re still on the screen by calling IsFullyVisibleFrom() which will count the corners via CountCornersVisibleFrom(). The moment the screen isn’t fully visible, we would switch to the NotInScreen state.
3. You can read more about these 2 functions from where it was shared, but essentially, all we’re doing is checking if the corners of our menu are on the player’s screen. If they are, then we return true, if not, we return false.
4. In Update(), when we’re in the NetScreen state, we switch to the Moving state and start a coroutine MoveToFrontOfPlayer() to move our menu to the front of the player. What’s interesting to note here is that I chose to use a coroutine, but we could have just as easily moved everything into our switch statement in Update() and we could have done almost the same thing.
5. In MoveToFrontOfPlayer(), we want to move our menu to be in front of the player, specifically our camera’s forward. To achieve this, we check its current position relative to our goal location which we create from GetCameraFoward(). If we’re not at the position, we would always set the rotation of the menu to be the same as the camera so it doesn’t look weird and use Vector3.MoveTowards to give us a position between our current position and our goal position. Then we yield and wait until the next frame. Once the menu goes to the front of the player, we change back to the InScreen state and go back to step 2.
6. In GetCameraFoward(), we set the location of where we want the menu to go to. Specifically, right in front of us. Luckily, this time we don’t have to do any trigonometry to get that specific location, we just need to get our camera’s forward and move it further out. We also need to adjust the y position to make sure that our menu also maintains the same y position we started at.

And that’s it for getting our camera to follow our head in VR!

Surprisingly, it’s not as difficult as we might have initially thought.

Here’s what it looks like:

We could have gradually rotated our menu if we wanted to, but I’ll leave that as something you can implement if you’re curious about how to do it.

Conclusion

With this finished, we have pretty much-implemented everything we wanted to regarding the menu.

However, before we finish, there are 2 important things that were left out:

• How to create a menu and…
• How to get rid of the menu!

In the next post, let’s look at how we can instantiate the menu and get rid of it programmatically!