Designing hand-based interaction for precise and efficient object manipulation in Virtual Reality

Abstract

Perceiving 3D anatomical data on a 2D screen is complicated, error-prone, and re- quires training for medical professionals because they must mentally reconstruct 3D data from 2D. When they perceive this 3D data in virtual reality, it reduces the 3D to 2D information loss, operating time, and cognitive load. Research in virtual reality applications for jaw osteotomy operation planning has explored interaction techniques using tracker, 3D pen, and haptic pen. However, medical professionals would like to use their hands as they do not need additional hardware and learn how to use it. Using hands as an input in virtual reality can be challenging because of noisy hand tracking caused by occlusion, head egomotion, lack of field of view (FOV), and lack of haptic feedback. In the process of jaw osteotomy operation planning, maximum accuracy is required for adjusting the osteotomy plane because this is the final step, and it compensates for the errors in the previous steps. This work focuses on designing and evaluating precise and efficient hand-based interaction techniques for plane alignment in virtual reality. In the design process, a contextual inquiry is conducted to understand the task. Then, literature review of hand-based object manipulation interaction techniques in virtual reality was conducted to create a tax- onomy of design factors. Then, potential design factors for hand-based interaction techniques were selected, based on which two interaction techniques were designed and further refined using pilot tests. A controlled experiment with 12 participants was conducted, to evaluate these two interaction techniques of (1) push and poke and (2) custom axis with C/D gain, for the plane alignment task using pinch-based direct manipulation as a baseline. From the study, it was found that push and poke was subjectively ranked more precise and preferred because it was faster, easy to learn and use and participants are confident using it. Based on the results of the study, design implications for future hand-based interaction techniques for precise plane alignment in virtual reality are discussed.