SMARTANNOTATOR: An Interactive Tool for Annotating Indoor RGBD Images

1National Tsing Hua University     2University College London    

Eurographics 2015


RGBD images with high quality annotations, both in the form of geometric (i.e., segmentation) and structural (i.e., how do the segments mutually relate in 3D) information, provide valuable priors for a diverse range of applications in scene understanding and image manipulation. While it is now simple to acquire RGBD images, annotating them, automatically or manually, remains challenging. We present SMARTANNOTATOR, an interactive system to facilitate annotating raw RGBD images. The system performs the tedious tasks of grouping pixels, creating potential abstracted cuboids, inferring object interactions in 3D, and generates an ordered list of hypotheses. The user simply has to flip through the suggestions for segment labels, finalize a selection, and the system updates the remaining hypotheses. As annotations are finalized, the process becomes simpler with fewer ambiguities to resolve. Moreover, as more scenes are annotated, the system makes better suggestions based on the structural and geometric priors learned from previous annotation sessions. We test the system on a large number of indoor scenes across different users and experimental settings, validate the results on existing benchmark datasets, and report significant improvements over low-level annotation alternatives.

  author = "Yu-Shiang Wong and Hung-Kuo Chu and Niloy J. Mitra",
  title = "SMARTANNOTATOR An Interactive Tool for Annotating Indoor RGBD Images",
  journal = "Computer Graphics Forum (Special issue of Eurographics 2015)",
  year = "2015"

Paper (20MB)


Code / Data
(12 GB)