ILLUSIVE – FOUNDATIONS OF PERCEPTION ENGINEERING

PRINCIPAL INVESTIGATOR
Professor Steven M. LaValle

FUNDING BODY
European Research Council (ERC-ADG-2020, grant number 101020977)

GRANT AMOUNT
2’496’578 EUR

TIMESPAN
8/2021 – 7/2026 (60 months)

ABSTRACT
Virtual reality (VR) technology has enormous potential to transform society, especially as the world faces unprecedented challenges of remote work and social distancing brought on by COVID 19. VR creates perceptual illusions that can uniquely enhance education, collaborative design, health care, and social interaction, all from a distance. Further benefits include highly immersive computer interfaces, data visualization, and storytelling.

In spite of its potential, the VR industry has struggled through hype cycles as devices built from co-opted component technologies fail to meet the expectations of each generation. As an industry leader in Huawei and early Oculus VR, the PI concluded that the struggle is largely due to the lack of a rigorous foundation that draws from the human-centered sciences of perceptual psychology, neuroscience, and physiology. The PI proposes to reduce the gap between engineering and these sciences by forging a new discipline called
perception engineering. In this pioneering view, the object being engineered is the perceptual illusion itself, and the physical devices that achieve it are auxiliary.

The project develops mathematical foundations for perception engineering that unify engineering concepts from robotics and control theory with their biological-system counterparts, especially predictive coding and the free energy principle. A cornerstone is von Neumann-Morgenstern information spaces, introduced for games with hidden information, which the PI has leveraged for fundamental problems in robotics throughout his research career. The project team includes experts from VR, neuroscience, perceptual psychology, robotics, control theory, and pure mathematics. The work lays a valuable foundation for principled engineering approaches to design, simulation, prediction, and analysis of sustained, targeted perceptual experiences. This offers valuable guidance and deeper insights into VR, robotics, and possibly the sciences that study perception.

SELECTED PUBLICATIONS
Arora N, Suomalainen M, Pouke M, Center EG, Mimnaugh KJ, Chambers AP, Pouke S & LaValle SM (2022) Augmenting immersive telepresence experience with a virtual body. IEEE Transactions on Visualization and Computer Graphics 28(5):2135-2145. https://doi.org/10.1109/TVCG.2022.3150473.

Suomalainen M, Sakcak B, Widagdo A, Kalliokoski J, Mimnaugh KJ, Chambers AP, Ojala T & LaValle SM (2022) Unwinding rotations improves user comfort with immersive telepresence robots. ACM/IEEE International Conference on Human-Robot Interaction (HRI 2022), Sapporo, Japan, in press.

EU & ERC