Multivariate statistical tests for comparing classification algorithms

Basit Öğe Kaydı
dc.authorid0000-0001-5838-4615
dc.authorid0000-0001-7506-0321
dc.contributor.authorYıldız, Olcay Taneren_US
dc.contributor.authorAslan, Özlemen_US
dc.contributor.authorAlpaydın, Ahmet İbrahim Ethemen_US
dc.date.accessioned2019-08-31T12:10:23Z
dc.date.accessioned2019-08-05T16:04:58Z
dc.date.available2019-08-31T12:10:23Z
dc.date.available2019-08-05T16:04:58Z
dc.date.issued2011
dc.departmentIşık Üniversitesi, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümüen_US
dc.departmentIşık University, Faculty of Engineering, Department of Computer Engineeringen_US
dc.description.abstractThe misclassification error which is usually used in tests to compare classification algorithms, does not make a distinction between the sources of error, namely, false positives and false negatives. Instead of summing these in a single number, we propose to collect multivariate statistics and use multivariate tests on them. Information retrieval uses the measures of precision and recall, and signal detection uses true positive rate (tpr) and false positive rate (fpr) and a multivariate test can also use such two values instead of combining them in a single value, such as error or average precision. For example, we can have bivariate tests for (precision, recall) or (tpr, fpr). We propose to use the pairwise test based on Hotelling's multivariate T test to compare two algorithms or multivariate analysis of variance (MANOVA) to compare L > 2 algorithms. In our experiments, we show that the multivariate tests have higher power than the univariate error test, that is, they can detect differences that the error test cannot, and we also discuss how the decisions made by different multivariate tests differ, to be able to point out where to use which. We also show how multivariate or univariate pairwise tests can be used as post-hoc tests after MANOVA to find cliques of algorithms, or order them along separate dimensions.en_US
dc.description.versionPublisher's Versionen_US
dc.identifier.citationYıldız O.T., Aslan Ö. & Alpaydın A. İ. E. (2011). Multivariate Statistical Tests for Comparing Classification Algorithms. In: Coello C.A.C. (eds) Learning and Intelligent Optimization. LION 2011. Paper present at the Lecture Notes in Computer Science, 6683, 1-15. doi:10.1007/978-3-642-25566-3_1en_US
dc.identifier.doi10.1007/978-3-642-25566-3_1
dc.identifier.endpage15
dc.identifier.isbn9783642255656
dc.identifier.isbn9783642255663
dc.identifier.issn0302-9743
dc.identifier.scopus2-s2.0-84868547148
dc.identifier.scopusqualityQ3
dc.identifier.startpage1
dc.identifier.urihttps://hdl.handle.net/11729/1939
dc.identifier.urihttps://dx.doi.org/10.1007/978-3-642-25566-3_1
dc.identifier.volume6683
dc.indekslendigikaynakScopusen_US
dc.institutionauthorYıldız, Olcay Taneren_US
dc.institutionauthorid0000-0001-5838-4615
dc.language.isoenen_US
dc.peerreviewedYesen_US
dc.publicationstatusPublisheden_US
dc.publisherSpringer, Berlin, Heidelbergen_US
dc.relation.ispartofLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)en_US
dc.relation.publicationcategoryKonferans Öğesi - Uluslararası - Kurum Öğretim Elemanıen_US
dc.rightsinfo:eu-repo/semantics/closedAccessen_US
dc.subjectNull hypothesisen_US
dc.subjectConfusion matrixen_US
dc.subjectAverage precisionen_US
dc.subjectUnivariate testen_US
dc.subjectMultivariate testen_US
dc.titleMultivariate statistical tests for comparing classification algorithmsen_US
dc.typeConference Objecten_US
dspace.entity.typePublication

Dosyalar

Orijinal paket
Listeleniyor 1 - 1 / 1
Küçük Resim Yok
İsim:
1939.pdf
Boyut:
790.03 KB
Biçim:
Adobe Portable Document Format
Açıklama:
Publisher's Version
İndir

Koleksiyon

Bildiri Koleksiyonu | Bilgisayar Mühendisliği Bölümü
Scopus İndeksli Yayınlar Koleksiyonu