SonalShooter: a spatial augmented reality system using handheld directional speaker with camera

Mainly in the field of Augmented Reality (AR), various systems that can overlap virtual information onto physical space in the real world have been developed. In such systems, wearing special equipment might obstruct users' natural interaction and communication. Contrarily, Spatial Augmented Reality, which project additional information directly on the targeted physical space, has attracted much attention. In our research, we aim to provide a novel auditory interaction with physical surfaces through the approach of Spatial Augmented Reality. Our system, named SonalShooter, can give auditory information to physical objects (Figure 1). When a user aims the device at a physical object, the device detects and identifies the target with a computer vision approach and he/she will be able to hear corresponding auditory information from the object itself as if it is speaking.