We’ve come a long way from using a mouse to navigate a screen, to pinching and zooming to navigate the same content on a smaller glass screen. Yet, the principle and how we do it don’t differ that much. Moving towards a 3D world is some whole other story. From points, clicks and touch screens to virtual interfaces in space: our interaction with technology is evolving from tangible to intangible. What underlies this evolution? Let’s travel back in time for a moment.

First stop: the 1990s. Man meets the Internet, a new connected world. We learn about computers, the mouse and graphical user interfaces (GUI), the first graphical interfaces to replace textual commands. We travel on to the year 2007: enter the iPhone. Smartphones had existed for some time, but the iPhone introduced a new design language. Pinching, zooming, swiping was initially very unusual; today it is the standard.

New modes of interaction are taking shape, but how do we handle them? Is your body the controller in new technologies like augmented and virtual reality. Do the design languages we know from desktop and mobile apps also work for these technologies? In this article I like to zoom in on seven challenges and tips for creating a user-friendly user interface for augmented & virtual reality applications.

1. Design for context and environment.

Unlike desktop and mobile, AR and VR use a 360° environment, although users are still strongly focused on the physical 2D screenspaces they know from desktop and mobile. The UI of AR/VR must make the user aware that an entire space exists with which to interact. We speak of environmental design. Amid an environment brimming with information, users are looking for relevant content more than before.

Tip:

Build your screen carefully, avoiding an overload of information and making sure it is adapted to the context in which a user is located. Provide an intuitive UI, easily accessible and without getting in the way of the user’s view.

2. Interaction.

Interaction design is becoming increasingly important. Interacting with 3D requires an entirely different approach because of the extra dimension. The user lets go of a tangible thing like a screen, to interact with movements, voice or via a controller.

Tip:

If you work with buttons in the UI, provide recognition, for example with elements from mobile applications. After all, we all fall back on things that are familiar to us. Too much cognitive baggage prevents a good user experience. We find that it is useful to offer the user some comfort by adopting familiar UI elements, if only for the transition phase

3. Visual and auditory cues.

Visual and auditory cues are useful to give users an indication or direction to a VR or AR experience. These can be cues like an arrow that makes the user look in a certain direction or auditory cues like a ping signal every time the user performs a correct action. They help the user better understand the context.

Tip:

Use these cues deliberately, because like visual elements, they are also part of your UI and can overload the interface.

4. Color and typography.

Color and typography work the same in all media. Even in augmented and virtual reality, we use conventional colors. Text gets a little more attention in AR/VR applications. VR headsets do not necessarily have the same resolution as desktop screens, which makes text more difficult to read at greater distances. Fortunately, technology is improving by the day and, in the meantime, we can look forward to better screens. For example, HTC Vive Pro’s resolution is 76% better than its predecessor, resulting in a much sharper image.

Tip:

As for color, use the conventions, green for an affirmative action, red for a threatening or negative action and blue as a calming or neutral color. AR/VR is entirely focused on an even tighter and more intuitive UI, we choose cues rather than text. Definitely avoid long text unless justified to clarify a specific context.

5. Material design as a design language.

When iOS came up with flat design, we as designers were elated. In the meantime, we have moved away from that as well and are focusing more on languages such as material design, which takes more account of natural layers. Such design language also lends itself more to AR/VR.

Tip:

Texture is becoming more important because we want to give the user the feeling that the 3D objects are real and tangible. We use light, weight and shadow to build the experience as realistically as possible.

6. Designing for the right device.

How the user interacts with an application depends heavily on the hardware chosen. Applications on smartphones react differently than on VR devices. The purpose of the application will also always determine which device to design for. Is the purpose to introduce anyone worldwide to a virtual environment or are you welcoming the user to your own showroom, that affects all of your design.

Tip:

Make sure a user positions the device correctly. A frequently used menu of options is best placed where it is easily accessible, yet not distracting. The size of a device also plays an important role. Because of this, designing for different devices means a lot of experimentation. After all, there are no general conventions yet.

7. Innovate where you can.

Conventions make it easy for a user to navigate through an interface. They unburden the user when operating an application. You can innovate, but only if it does not interfere with the user’s convenience. The same is true of AR and VR, although these technologies lend themselves a little more to innovation at this time.

AR and VR are relatively new technologies and designing in function of these technologies still involves a lot of trial and error. We search for what works by frequent testing and gathering user feedback. Over time, the feedback converts into conventions.