Dynamic weighted majority: a new ensemble method for tracking concept drift
- 23 April 2004
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 11, 123-130
- https://doi.org/10.1109/icdm.2003.1250911
Abstract
Algorithms for tracking concept drift are important for many applications. We present a general method based on the weighted majority algorithm for using any online learner for concept drift. Dynamic weighted majority (DWM) maintains an ensemble of base learners, predicts using a weighted-majority vote of these "experts", and dynamically creates and deletes experts in response to changes in performance. We empirically evaluated two experimental systems based on the method using incremental naive Bayes and incremental tree inducer [ITI] as experts. For the sake of comparison, we also included Blum's implementation of weighted majority. On the STAGGER concepts and on the SEA concepts, results suggest that the ensemble method learns drifting concepts almost as well as the base algorithms learn each concept individually. Indeed, we report the best overall results for these problems to date.Keywords
This publication has 17 references indexed in Scilit:
- Incremental rule learning with partial instance memory for changing conceptsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- Mining time-changing data streamsPublished by Association for Computing Machinery (ACM) ,2001
- A streaming ensemble algorithm (SEA) for large-scale classificationPublished by Association for Computing Machinery (ACM) ,2001
- Mining high-speed data streamsPublished by Association for Computing Machinery (ACM) ,2000
- The application of AdaBoost for distributed, scalable and on-line learningPublished by Association for Computing Machinery (ACM) ,1999
- Bagging predictorsMachine Learning, 1996
- Learning in the presence of concept drift and hidden contextsMachine Learning, 1996
- The Weighted Majority AlgorithmInformation and Computation, 1994
- Learning quickly when irrelevant attributes abound: A new linear-threshold algorithmMachine Learning, 1988
- Probability Inequalities for Sums of Bounded Random VariablesJournal of the American Statistical Association, 1963