Identification of plants from multiple images and botanical IdKeys

Automatic retrieval tools are becoming increasingly important in botany and agriculture due to the growing interest in biodiversity and the ongoing shortage of skilled taxonomists. Our work is motivated by a botanical field scenario where the basic unit of observation is a plant. We describe a novel, image-based retrieval system for both educational and decision-making purposes. Given multiple leaf images of the same plant, the algorithm displays a ranked list of the most relevant species, along with a varied set of representative images from each estimated species. We focus on leaves but the strategy is generic, based on a hierarchical representation of latent variables called identification keys (IdKeys) which embody domain knowledge about taxonomy and landmarks. For each query image, keys are estimated sequentially, proceeding from landmarks to the genus and finally to an estimated set of species. The results over multiple queries are then collated into a single ranked list of species. Experiments demonstrate that the proposed approach achieves excellent performance on several databases of uncluttered leaf images as well as providing an instructive interface for measuring diversity and identifying new species.