Touchless Tactile Interaction with Unconventional Permeable Displays

Abstract

Unconventional displays, such as 3D displays, projection screens formed of flowing light-scattering particles (fogscreens), and virtual reality (VR) headsets, can create illusions of images floating in mid-air. Paired with hand tracking, gestural interaction with floating user interfaces (UI) is possible on this permeable imagery, thus creating reach-through touchscreens that react and recover instantly from intersecting fingers and objects. The user can explore virtual environments and control floating UIs with hand gestures which could help, for example, in simulated training and in creating an improved feeling of immersion. However, hand-based gestural interaction with such UIs can be difficult without haptic sensations typical in daily activities. Without haptics, the level of immersion and smoothness of interaction suffers if the hands can pass through virtual objects without triggering tactile sensations. Ultrasound haptics is a method to produce a focused airborne acoustic air pressure on a user’s skin, thus creating an unobtrusive, mid-air sensation of touch. Fogscreens, VR headsets, or some other unconventional displays together with ultrasound haptics enable tactile interaction with “touchless touchscreens”. These tactile, floating UIs open new opportunities, e.g., for immersive interaction, advertisement, and entertainment. It can bring back the missing haptic feedback for these displays.