An Occlusion and Pose Sensitive Image Dataset for Black Ear Recognition

RESEARCH APPROACH

The research approach adopted for the study consists of seven phases which includes as shown in Figure 1:

1. Pre-acquisition
2. data pre-processing
3. Raw images collection
4. Image pre-processing
5. Naming of images
6. Dataset Repository
7. Performance Evaluation

The different phases in the study are discussed in the sections below.

 

PRE-ACQUISITION

The volunteers are given brief orientation on how their data will be managed and used for research purposes only. After the volunteers agrees, a consent form is given to be read and signed. The sample of the consent form filled by the volunteers is shown in Figure 1.

The capturing of images was started with the setup of the imaging device. The camera is set up on a tripod stand in stationary position at the height 90 from the floor and distance 20cm from the subject.

 

EAR IMAGE ACQUISITION

Image acquisition is an action of retrieving image from an external source for further processing. The image acquisition is purely a hardware dependent process by capturing unprocessed images of the volunteers using a professional camera. This was acquired through a subject posing in front of the camera. It is also a process through which digital representation of a scene can be obtained. This representation is known as an image and its elements are called pixels (picture elements). The imaging sensor/camera used in this study is a Canon E0S 60D professional camera which is placed at a distance of 3 feet form the subject and 20m from the ground. 

This is the first step in this project to achieve the project’s aim of developing an occlusion and pose sensitive image dataset for black ear recognition. (OPIB ear dataset). To achieve the objectives of this study, a set of black ear images were collected mostly from undergraduate students at a public University in Nigeria.

 

The image dataset required is captured in two scenarios:

1. uncontrolled environment with a surveillance camera

The image dataset captured is purely black ear with partial occlusion in a constrained and unconstrained environment.

 

2. controlled environment with professional cameras

The ear images captured were from black subjects in controlled environment. To make the OPIB dataset pose invariant, the volunteers stand on a marked positions on the floor indicating the angles at which the imaging sensor was captured the volunteers’ ear. The capturing of the images in this category requires that the subject stand and rotates in the following angles 60^o, 30^o and 0^o towards their right side to capture the left ear and then towards the left to capture the right ear (Fernando et al., 2017) as shown in Figure 4. Six (6) images were captured per subject at angles 60^o, 30^o and 0^o for the left and right ears of 152 volunteers making a total of 907 images (five volunteers had 5 images instead of 6, hence folders 34, 22, 51, 99 and 102 contain 5 images).

To make the OPIB dataset occlusion and pose sensitive, partial occlusion of the subject’s ears were simulated using rings, hearing aid, scarf, earphone/ear pods, etc. before the images are captured.

 

CONSENT FORM This form was designed to obtain participant’s consent on the project titled: An Occlusion and Pose Sensitive Image Dataset for Black Ear Recognition (OPIB). The information is purely needed for academic research purposes and the ear images collected will curated anonymously and the identity of the volunteers will not be shared with anyone. The images will be uploaded on online repository to aid research in ear biometrics. The participation is voluntary, and the participant can withdraw from the project any time before the final dataset is curated and warehoused. Kindly sign the form to signify your consent. I consent to my image being recorded in form of still images or video surveillance as part of the OPIB ear images project. Tick as appropriate: GENDER       Male    Female AGE               (18-25)                        (26-35)                        (36-50)   ……………………………….. SIGNED

 

Figure 1: Sample of Subject’s Consent Form for the OPIB ear dataset

 

RAW IMAGE COLLECTION

The ear images were captured using a digital camera which was set to JPEG because if the camera format is set to raw, no processing will be applied, hence the stored file will contain more tonal and colour data. However, if set to JPEG, the image data will be processed, compressed and stored in the appropriate folders.

 

IMAGE PRE-PROCESSING

The aim of pre-processing is to improve the quality of the images with regards to contrast, brightness and other metrics. It also includes operations such as: cropping, resizing, rescaling, etc. which are important aspect of image analysis aimed at dimensionality reduction. The images are downloaded on a laptop for processing using MATLAB.

 

Image Cropping

The first step in image pre-processing is image cropping. Some irrelevant parts of the image can be removed, and the image Region of Interest (ROI) is focused. This tool provides a user with the size information of the cropped image. MATLAB function for image cropping realizes this operation interactively by waiting for a user to specify the crop rectangle with the mouse and operate on the current axes. The output images of the cropping process are of the same class as the input image.

Naming of OPIB Ear Images

The OPIB ear images were labelled based on the naming convention formulated from this study as shown in Figure 5. The images are given unique names that specifies the subject, the side of the ear (left or right) and the angle of capture. The first and second letters (SU) in the image names is block letter simply representing subject for subject 1-to-n in the dataset, while the left and right ears is distinguished using L1, L2, L3 and R1, R2, R3 for angles 60⁰, 30⁰ and 0⁰_, respectively as shown in Table 1.

 

Table 1: Naming Convention for OPIB ear images

NAMING CONVENTION

Label Degrees                                     600     300      00

No of the degree                     1          2          3

Subject 1   indicates          (first image in dataset) SU1

Subject n   indicates          (last image in dataset) SUn

Left Image 1                                 L 1 Left image n                                 L n Right Image 1                               R 1 Right Image n                               R n

SU1L1                                           SU1RI SU1L2                                           SU1R2             SU1L3                                           SU1R3

 

OPIB EAR DATASET EVALUATION

The prominent challenges with the current evaluation practices in the field of ear biometrics are the use of different databases, different evaluation matrices, different classifiers that mask the feature extraction performance and the time spent developing framework (Abaza et al., 2013; Emeršič et al., 2017).

The toolbox provides environment in which the evaluation of methods for person recognition based on ear biometric data is simplified. It executes all the dataset reads and classification based on ear descriptors.

 

DESCRIPTION OF OPIB EAR DATASET

OPIB ear dataset was organised into a structure with each folder containing 6 images of the same person. The images were captured with both left and right ear at angle 0, 30 and 60 degrees. The images were occluded with earing, scarves and headphone etc.  The collection of the dataset was done both indoor and outdoor.  The dataset was gathered through the student at a public university in Nigeria. The percentage of female (40.35%) while Male (59.65%).  The ear dataset was captured through a profession camera Nikon D 350. It was set-up with a camera stand where an individual captured in a process order. A total number of 907 images was gathered.

The challenges encountered in term of gathering students for capturing, processing of the images and annotations. The volunteers were given a brief orientation on what their ear could be used for before, it was captured, for processing.  It was a great task in arranging the ear (dataset) into folders and naming accordingly.

 

Table 2: Overview of the OPIB Ear Dataset

Location	Both Indoor and outdoor environment
Information about Volunteers	Students
Gender	Female (40.35%) and male (59.65%)
Head Side Left and Right	Side Left and Right
Total number of volunteers	152
Per Subject images	3 images of left ear and 3 images of right ear
Total Images	907
Age group	18 to 35 years
Colour Representation	RGB
Image Resolution	224x224