Strategies for Combining Tree-Based Ensemble Models

• Main Author: Zhang, Yi
• Format: Others
• Published: NSUWorks 2017
• Subjects: ensemble models; model selection; multiple correspondence analysis; predictive models; random forest; extremely randomized tree; and eXtreme gradient boosting model; tree based ensemble model; Computer Sciences
• Online Access: http://nsuworks.nova.edu/gscis_etd/1021; http://nsuworks.nova.edu/cgi/viewcontent.cgi?article=2019&context=gscis_etd

Ensemble models have proved effective in a variety of classification tasks. These models combine the predictions of several base models to achieve higher out-of-sample classification accuracy than the base models. Base models are typically trained using different subsets of training examples and input features. Ensemble classifiers are particularly effective when their constituent base models are diverse in terms of their prediction accuracy in different regions of the feature space. This dissertation investigated methods for combining ensemble models, treating them as base models. The goal is to develop a strategy for combining ensemble classifiers that results in higher classification accuracy than the constituent ensemble models. Three of the best performing tree-based ensemble methods – random forest, extremely randomized tree, and eXtreme gradient boosting model – were used to generate a set of base models. Outputs from classifiers generated by these methods were then combined to create an ensemble classifier. This dissertation systematically investigated methods for (1) selecting a set of diverse base models, and (2) combining the selected base models. The methods were evaluated using public domain data sets which have been extensively used for benchmarking classification models. The research established that applying random forest as the final ensemble method to integrate selected base models and factor scores of multiple correspondence analysis turned out to be the best ensemble approach.