Design and analysis of classifier learning experiments in bioinformatics: survey and case studies
Yükleniyor...
Tarih
2012-12
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
IEEE Computer Soc
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
In many bioinformatics applications, it is important to assess and compare the performances of algorithms trained from data, to be able to draw conclusions unaffected by chance and are therefore significant. Both the design of such experiments and the analysis of the resulting data using statistical tests should be done carefully for the results to carry significance. In this paper, we first review the performance measures used in classification, the basics of experiment design and statistical tests. We then give the results of our survey over 1,500 papers published in the last two years in three bioinformatics journals (including this one). Although the basics of experiment design are well understood, such as resampling instead of using a single training set and the use of different performance metrics instead of error, only 21 percent of the papers use any statistical test for comparison. In the third part, we analyze four different scenarios which we encounter frequently in the bioinformatics literature, discussing the proper statistical methodology as well as showing an example case study for each. With the supplementary software, we hope that the guidelines we discuss will play an important role in future studies.
Açıklama
Anahtar Kelimeler
Statistical tests, Classification, Model selection, Multiple data sets, Statistical comparisons, Roc curve, Algorithms, Precision, Performance, Prediction, Retrieval, Recall, Area
Kaynak
IEEE/ACM Transactions on Computational Biology and Bioinformatics
WoS Q Değeri
Q1
Scopus Q Değeri
Q2
Cilt
9
Sayı
Künye
İrsoy, O., Yıldız, O. T. & Alpaydın, A. İ. E. (2012). Design and analysis of classifier learning experiments in bioinformatics: Survey and case studies. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 9(6), 1663-1675. doi:10.1109/TCBB.2012.117
