Optimal Feature Set Size in Random Forest Regression

• Main Authors: Sunwoo Han, Hyunjoong Kim
• Format: Article
• Language: English
• Published: MDPI AG 2021-04-01
• Series: Applied Sciences
• Subjects: random forest; feature set size; grid search; regression
• Online Access: https://www.mdpi.com/2076-3417/11/8/3428

One of the most important hyper-parameters in the Random Forest (RF) algorithm is the feature set size used to search for the best partitioning rule at each node of trees. Most existing research on feature set size has been done primarily with a focus on classification problems. We studied the effect of feature set size in the context of regression. Through experimental studies using many datasets, we first investigated whether the RF regression predictions are affected by the feature set size. Then, we found a rule associated with the optimal size based on the characteristics of each data. Lastly, we developed a search algorithm for estimating the best feature set size in RF regression. We showed that the proposed search algorithm can provide improvements over other choices, such as using the default size specified in the <i>randomForest</i> R package and using the common grid search method.