Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models
The scientific evaluation methodology for the forecast accuracy of wind power forecasting models is an important issue in the domain of wind power forecasting. However, traditional forecast evaluation criteria, such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), have limitations in appli...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2014-07-01
|
Series: | Energies |
Subjects: | |
Online Access: | http://www.mdpi.com/1996-1073/7/7/4185 |
id |
doaj-c147eefb611a40708a04384866ad8262 |
---|---|
record_format |
Article |
spelling |
doaj-c147eefb611a40708a04384866ad82622020-11-24T22:30:01ZengMDPI AGEnergies1996-10732014-07-01774185419810.3390/en7074185en7074185Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting ModelsHao Chen0Qiulan Wan1Yurong Wang2School of Electrical Engineering, Southeast University, No.2 Sipailou, Nanjing 210096, ChinaSchool of Electrical Engineering, Southeast University, No.2 Sipailou, Nanjing 210096, ChinaSchool of Electrical Engineering, Southeast University, No.2 Sipailou, Nanjing 210096, ChinaThe scientific evaluation methodology for the forecast accuracy of wind power forecasting models is an important issue in the domain of wind power forecasting. However, traditional forecast evaluation criteria, such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), have limitations in application to some degree. In this paper, a modern evaluation criterion, the Diebold-Mariano (DM) test, is introduced. The DM test can discriminate the significant differences of forecasting accuracy between different models based on the scheme of quantitative analysis. Furthermore, the augmented DM test with rolling windows approach is proposed to give a more strict forecasting evaluation. By extending the loss function to an asymmetric structure, the asymmetric DM test is proposed. Case study indicates that the evaluation criteria based on DM test can relieve the influence of random sample disturbance. Moreover, the proposed augmented DM test can provide more evidence when the cost of changing models is expensive, and the proposed asymmetric DM test can add in the asymmetric factor, and provide practical evaluation of wind power forecasting models. It is concluded that the two refined DM tests can provide reference to the comprehensive evaluation for wind power forecasting models.http://www.mdpi.com/1996-1073/7/7/4185wind power forecasting evaluationloss functionDiebold-Mariano (DM) testaugmented DM testasymmetric DM testevaluation criteria |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Hao Chen Qiulan Wan Yurong Wang |
spellingShingle |
Hao Chen Qiulan Wan Yurong Wang Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models Energies wind power forecasting evaluation loss function Diebold-Mariano (DM) test augmented DM test asymmetric DM test evaluation criteria |
author_facet |
Hao Chen Qiulan Wan Yurong Wang |
author_sort |
Hao Chen |
title |
Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models |
title_short |
Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models |
title_full |
Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models |
title_fullStr |
Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models |
title_full_unstemmed |
Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models |
title_sort |
refined diebold-mariano test methods for the evaluation of wind power forecasting models |
publisher |
MDPI AG |
series |
Energies |
issn |
1996-1073 |
publishDate |
2014-07-01 |
description |
The scientific evaluation methodology for the forecast accuracy of wind power forecasting models is an important issue in the domain of wind power forecasting. However, traditional forecast evaluation criteria, such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), have limitations in application to some degree. In this paper, a modern evaluation criterion, the Diebold-Mariano (DM) test, is introduced. The DM test can discriminate the significant differences of forecasting accuracy between different models based on the scheme of quantitative analysis. Furthermore, the augmented DM test with rolling windows approach is proposed to give a more strict forecasting evaluation. By extending the loss function to an asymmetric structure, the asymmetric DM test is proposed. Case study indicates that the evaluation criteria based on DM test can relieve the influence of random sample disturbance. Moreover, the proposed augmented DM test can provide more evidence when the cost of changing models is expensive, and the proposed asymmetric DM test can add in the asymmetric factor, and provide practical evaluation of wind power forecasting models. It is concluded that the two refined DM tests can provide reference to the comprehensive evaluation for wind power forecasting models. |
topic |
wind power forecasting evaluation loss function Diebold-Mariano (DM) test augmented DM test asymmetric DM test evaluation criteria |
url |
http://www.mdpi.com/1996-1073/7/7/4185 |
work_keys_str_mv |
AT haochen refineddieboldmarianotestmethodsfortheevaluationofwindpowerforecastingmodels AT qiulanwan refineddieboldmarianotestmethodsfortheevaluationofwindpowerforecastingmodels AT yurongwang refineddieboldmarianotestmethodsfortheevaluationofwindpowerforecastingmodels |
_version_ |
1725742209799553024 |