Self-paced Learning for Imbalanced Data

In this paper, we propose a novel training paradigm that combines two learning strategies: cost-sensitive and self-paced learning. This learning approach can be applied to the decision problems where highly imbalanced data is used during training process. The main idea behind the proposed method is to start the learning process by taking large number of minority examples and only the easiest majority objects and then gradually turning to more difficult cases. We examine the quality of this training paradigm comparing to other learning schemas for neural network model using a set of highly imbalanced benchmark datasets.

1. 1.

   We consider two-class imbalanced data problem in which the minority class is assumed to be positive and the majority class is associated with the negative class.

2. 2.

   Absolute value can be omitted since \(\mathbf {v} \in \{0,1\}^n\).

3. 3.

   We can always utilize about 10–20% data for validation.

4. 4.

   AUC is defined as the arithmetic mean of True Positive Rate (TPR, called Sensitivity)and the True Negative Rate (TNR, called Specificity), \(AUC = \frac{TPR+TNR}{2}\). TP,TN,FP,FN are the elements of the confusion matrix, \(TPR = \frac{TP}{TP+FN}\) and \(TNR = \frac{TN}{TN+FP}\). We can represent the AUC value in such form if we consider classes, not probabilities while testing. In such case the ROC curve is represented by one point located in position (TPR,FPR). The area under ROC curve can be calculated using the procedure \(AUC=\frac{1+TPR-FPR}{2}\). Making use of \(TNR=1-FPR\) we have \(AUC=\frac{1}{2}(TPR+TNR)\). In our opinion this method of calculating AUC is better for imbalanced data problems, because it evaluates true predictions instead of the ordering of data that is used for evaluation.

The research conducted by the authors has been partially co-financed by the Ministry of Science and Higher Education, Republic of Poland, namely, Maciej Zięba: grant No. B50083/W8/K3, Jakub M. Tomczak: grant No. B50106/W8/K3.

Authors and Affiliations

1. Department of Computer Science, Wroclaw University of Technology, Wybrzeze Wyspianskiego 27, 50-370, Wroclaw, Poland

   Maciej Zięba, Jakub M. Tomczak & Jerzy Świątek

Corresponding author

Correspondence to Maciej Zięba .

Editors and Affiliations

1. Wrocław University of Technology, Wrocław, Poland

   Ngoc Thanh Nguyen

2. Wrocław University of Technology, Wrocław, Poland

   Bogdan Trawiński

3. Iwate Prefectural University, Takizawa, Japan

   Hamido Fujita

4. National University of Kaohsiung, Kaohsiung, Taiwan

   Tzung-Pei Hong

Reprints and permissions

© 2016 Springer-Verlag Berlin Heidelberg

Policies and ethics