caa a22 4500 605478538 CHVBK 20210128100406.0 cr unu---uuuuu 210128e20150501xx s 000 0 eng 10.1007/s10994-014-5466-8 doi (NATIONALLICENCE)springer-10.1007/s10994-014-5466-8 Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions [Elektronische Daten] [Robert Durrant, Ata Kabán] We prove theoretical guarantees for an averaging-ensemble of randomly projected Fisher linear discriminant classifiers, focusing on the case when there are fewer training observations than data dimensions. The specific form and simplicity of this ensemble permits a direct and much more detailed analysis than existing generic tools in previous works. In particular, we are able to derive the exact form of the generalization error of our ensemble, conditional on the training set, and based on this we give theoretical guarantees which directly link the performance of the ensemble to that of the corresponding linear discriminant learned in the full data space. To the best of our knowledge these are the first theoretical results to prove such an explicit link for any classifier and classifier ensemble pair. Furthermore we show that the randomly projected ensemble is equivalent to implementing a sophisticated regularization scheme to the linear discriminant learned in the original data space and this prevents overfitting in conditions of small sample size where pseudo-inverse FLD learned in the data space is provably poor. Our ensemble is learned from a set of randomly projected representations of the original high dimensional data and therefore for this approach data can be collected, stored and processed in such a compressed form. We confirm our theoretical findings with experiments, and demonstrate the utility of our approach on several datasets from the bioinformatics domain and one very high dimensional dataset from the drug discovery domain, both settings in which fewer observations than dimensions are the norm. The Author(s), 2014 Random projections nationallicence Ensemble learning nationallicence Linear discriminant analysis nationallicence Compressed learning nationallicence Learning theory nationallicence Durrant Robert Department of Statistics, University of Waikato, 3240, Hamilton, New Zealand aut Kabán Ata School of Computer Science, University of Birmingham, B15 2TT, Edgbaston, UK aut Machine Learning Springer US; http://www.springer-ny.com 99/2(2015-05-01), 257-286 0885-6125 99:2<257 2015 99 10994 https://doi.org/10.1007/s10994-014-5466-8 text/html Onlinezugriff via DOI BK010053 XK010053 XK010000 Metadata rights reserved Springer special CC-BY-NC licence nationallicence 1 research-article jats NATIONALLICENCE NATIONALLICENCE NL-springer NATIONALLICENCE 856 40 https://doi.org/10.1007/s10994-014-5466-8 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Durrant Robert Department of Statistics, University of Waikato, 3240, Hamilton, New Zealand aut NATIONALLICENCE 700 1- Kabán Ata School of Computer Science, University of Birmingham, B15 2TT, Edgbaston, UK aut NATIONALLICENCE 773 0- Machine Learning Springer US; http://www.springer-ny.com 99/2(2015-05-01), 257-286 0885-6125 99:2<257 2015 99 10994