Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models

• Main Authors: Hao Chen, Qiulan Wan, Yurong Wang
• Format: Article
• Language: English
• Published: MDPI AG 2014-07-01
• Series: Energies
• Subjects: wind power forecasting evaluation; loss function; Diebold-Mariano (DM) test; augmented DM test; asymmetric DM test; evaluation criteria
• Online Access: http://www.mdpi.com/1996-1073/7/7/4185

The scientific evaluation methodology for the forecast accuracy of wind power forecasting models is an important issue in the domain of wind power forecasting. However, traditional forecast evaluation criteria, such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), have limitations in application to some degree. In this paper, a modern evaluation criterion, the Diebold-Mariano (DM) test, is introduced. The DM test can discriminate the significant differences of forecasting accuracy between different models based on the scheme of quantitative analysis. Furthermore, the augmented DM test with rolling windows approach is proposed to give a more strict forecasting evaluation. By extending the loss function to an asymmetric structure, the asymmetric DM test is proposed. Case study indicates that the evaluation criteria based on DM test can relieve the influence of random sample disturbance. Moreover, the proposed augmented DM test can provide more evidence when the cost of changing models is expensive, and the proposed asymmetric DM test can add in the asymmetric factor, and provide practical evaluation of wind power forecasting models. It is concluded that the two refined DM tests can provide reference to the comprehensive evaluation for wind power forecasting models.