caa a22 4500 605478007 CHVBK 20210128100403.0 cr unu---uuuuu 210128e20150301xx s 000 0 eng 10.1007/s10994-014-5441-4 doi (NATIONALLICENCE)springer-10.1007/s10994-014-5441-4 Evaluation methods and decision theory for classification of streaming data with temporal dependence [Elektronische Daten] [Indrė Žliobaitė, Albert Bifet, Jesse Read, Bernhard Pfahringer, Geoff Holmes] Predictive modeling on data streams plays an important role in modern data analysis, where data arrives continuously and needs to be mined in real time. In the stream setting the data distribution is often evolving over time, and models that update themselves during operation are becoming the state-of-the-art. This paper formalizes a learning and evaluation scheme of such predictive models. We theoretically analyze evaluation of classifiers on streaming data with temporal dependence. Our findings suggest that the commonly accepted data stream classification measures, such as classification accuracy and Kappa statistic, fail to diagnose cases of poor performance when temporal dependence is present, therefore they should not be used as sole performance indicators. Moreover, classification accuracy can be misleading if used as a proxy for evaluating change detectors with datasets that have temporal dependence. We formulate the decision theory for streaming data classification with temporal dependence and develop a new evaluation methodology for data stream classification that takes temporal dependence into account. We propose a combined measure for classification performance, that takes into account temporal dependence, and we recommend using it as the main performance measure in classification of streaming data. The Author(s), 2014 Data streams nationallicence Evaluation nationallicence Temporal dependence nationallicence Classification nationallicence Žliobaitė Indrė Department of Information and Computer Science, Aalto University and Helsinki Institute for Information Technology (HIIT), Espoo, Finland aut Bifet Albert Huawei Noah's Ark Research Lab, Hong Kong, China aut Read Jesse Department of Information and Computer Science, Aalto University and Helsinki Institute for Information Technology (HIIT), Espoo, Finland aut Pfahringer Bernhard University of Waikato, Hamilton, New Zealand aut Holmes Geoff University of Waikato, Hamilton, New Zealand aut Machine Learning Springer US; http://www.springer-ny.com 98/3(2015-03-01), 455-482 0885-6125 98:3<455 2015 98 10994 https://doi.org/10.1007/s10994-014-5441-4 text/html Onlinezugriff via DOI BK010053 XK010053 XK010000 Metadata rights reserved Springer special CC-BY-NC licence nationallicence 1 research-article jats NATIONALLICENCE NATIONALLICENCE NL-springer NATIONALLICENCE 856 40 https://doi.org/10.1007/s10994-014-5441-4 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Žliobaitė Indrė Department of Information and Computer Science, Aalto University and Helsinki Institute for Information Technology (HIIT), Espoo, Finland aut NATIONALLICENCE 700 1- Bifet Albert Huawei Noah's Ark Research Lab, Hong Kong, China aut NATIONALLICENCE 700 1- Read Jesse Department of Information and Computer Science, Aalto University and Helsinki Institute for Information Technology (HIIT), Espoo, Finland aut NATIONALLICENCE 700 1- Pfahringer Bernhard University of Waikato, Hamilton, New Zealand aut NATIONALLICENCE 700 1- Holmes Geoff University of Waikato, Hamilton, New Zealand aut NATIONALLICENCE 773 0- Machine Learning Springer US; http://www.springer-ny.com 98/3(2015-03-01), 455-482 0885-6125 98:3<455 2015 98 10994