EasyReg: Markerless Rigid Object Registration and Tracking Using All-In-One Mixed Reality Display

Overview:

I have designed an ICP-based algorithm to track rigid objects without fiducial or IR markers using mixed-reality or augmented-reality headset. This markerless tracking method was tested on Hololens 2 HMD, which has a ToF depth sensor. This work was submitted to ISMAR 2025 (first-author).

Why Important?

This markerless registration and tracking approach enables easy and accurate 3D registration in AR. It allows machines to perceive the location of physical objects accurately, unlocking various applications built upon this invention. One example is markerless tumor localization in the Operating Room. Similar systems are built by Apple, Meta for hand and object tracking.

Skills:

Computer Vision, Robot Perception, UWP