Augmented reality (AR) glasses are increasingly used by professionals such as surgeons, engineers, and field technicians to display digital information directly in their field of view. However, today’s optical see-through AR headsets still struggle to accurately align virtual overlays with the real world. Even minor misalignments between the user’s eyes and the headset’s cameras can cause discomfort, visual confusion, or even display incorrect information.

The project aims to tackle this fundamental challenge by rethinking how AR systems compute and display what the user actually sees. By developing new methods for “novel view synthesis” - the process of calculating what the world looks like from each of the user’s eyes - the research will enable a more precise and natural integration of digital content into physical environments. This could significantly improve the usability and reliability of future AR systems.

“To make AR truly useful and comfortable, digital information must seamlessly blend into our visual world – as if it were really there. Our goal is to bridge that gap by modelling how the world looks from the user’s own eyes, not just from the headset’s cameras,” says Professor Tobias Langlotz.

The project combines computer vision, neural rendering, and visual computing to build prototypes of AR glasses that can dynamically generate the user’s perspective in real time. If successful, the research could make AR applications more effective in fields such as medicine, remote maintenance, and education.