Stability of feature preference is a most vital yet rarely explored characteristics of feature ranking algorithms. In this study, 23 feature rankers are evaluated on 4 breast cancer datasets (BCDR-F03, WDBC, GSE10810 and GSE15852) using an advanced stability estimator (S), and 3 rankers are identified showing good stability (S ≥ 0.55) consistently on the four datasets. It suggests that data sufficiency is crucial for the construction of feature importance measure, since more rankers are stable on medical imaging datsets (BCDR-F03 and WDBC) than on gene expression datasets (GSE10810 and GSE15852), and high-dimensional small-sample-size datasets are big challenges of stability estimation. In our future work, more attention should be paid to the topics of developing stable feature ranking algorithms and stability estimators to well tackle different sizes of medical datasets.