Optimal estimator for assessing landslide model performance
The commonly used success rate (SR) in evaluating cell-based landslide model performance is based on the ratio of successfully predicted landslide sites over total actual landslide sites without considering the performance in predicting stable cells. We proposed a modified SR (MSR), in which the per...
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
Copernicus Publications
2006-01-01
|
Series: | Hydrology and Earth System Sciences |
Online Access: | http://www.hydrol-earth-syst-sci.net/10/957/2006/hess-10-957-2006.pdf |
id |
doaj-f305778f90614174906516f303670e19 |
---|---|
record_format |
Article |
spelling |
doaj-f305778f90614174906516f303670e192020-11-25T00:47:13ZengCopernicus PublicationsHydrology and Earth System Sciences1027-56061607-79382006-01-01106957965Optimal estimator for assessing landslide model performanceJ. C. HuangS. J. KaoThe commonly used success rate (SR) in evaluating cell-based landslide model performance is based on the ratio of successfully predicted landslide sites over total actual landslide sites without considering the performance in predicting stable cells. We proposed a modified SR (MSR), in which the performance of stable cell prediction is included. The advantage of MSR is to avoid over- and under-prediction while upholding the stable sensitivity throughout all simulated cases. Stochastic analyses are conducted by using artificial landslide maps and simulations with a full range of performances (from worst to perfect) in both stable and unstable cell predictions. Stochastic analyses reveal mathematical responses of estimators to various model results in calculating performance. The Kappa method, which is commonly used for satellite image analysis, is improper for landslide modeling giving inconsistent performance when landslide coverage changes. To examine differences among SR and MSR in real model application, we applied the SHALSTAB model onto a mountainous watershed in Taiwan. Case study shows that stable and unstable cell predictions are inter-exclusive in SHALSTAB model. The optimal estimator should compromise landslide over- and under-prediction. According to our 4000 simulations, the best simulation generated by MSR projects 83 hits over 131 actual landslide sites while the unstable cells cover only 16% of the studied watershed. By contrast, despite the fact that the best simulation deduced from SR projects 120 hits over 131 actual landslide sites, this high performance is only obtained when unstable cells cover an incredibly high landslide cover (~75%) of the entire watershed exhibiting a significant landslide over-prediction.http://www.hydrol-earth-syst-sci.net/10/957/2006/hess-10-957-2006.pdf |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
J. C. Huang S. J. Kao |
spellingShingle |
J. C. Huang S. J. Kao Optimal estimator for assessing landslide model performance Hydrology and Earth System Sciences |
author_facet |
J. C. Huang S. J. Kao |
author_sort |
J. C. Huang |
title |
Optimal estimator for assessing landslide model performance |
title_short |
Optimal estimator for assessing landslide model performance |
title_full |
Optimal estimator for assessing landslide model performance |
title_fullStr |
Optimal estimator for assessing landslide model performance |
title_full_unstemmed |
Optimal estimator for assessing landslide model performance |
title_sort |
optimal estimator for assessing landslide model performance |
publisher |
Copernicus Publications |
series |
Hydrology and Earth System Sciences |
issn |
1027-5606 1607-7938 |
publishDate |
2006-01-01 |
description |
The commonly used success rate (SR) in evaluating cell-based landslide model performance is based on the ratio of successfully predicted landslide sites over total actual landslide sites without considering the performance in predicting stable cells. We proposed a modified SR (MSR), in which the performance of stable cell prediction is included. The advantage of MSR is to avoid over- and under-prediction while upholding the stable sensitivity throughout all simulated cases. Stochastic analyses are conducted by using artificial landslide maps and simulations with a full range of performances (from worst to perfect) in both stable and unstable cell predictions. Stochastic analyses reveal mathematical responses of estimators to various model results in calculating performance. The Kappa method, which is commonly used for satellite image analysis, is improper for landslide modeling giving inconsistent performance when landslide coverage changes. To examine differences among SR and MSR in real model application, we applied the SHALSTAB model onto a mountainous watershed in Taiwan. Case study shows that stable and unstable cell predictions are inter-exclusive in SHALSTAB model. The optimal estimator should compromise landslide over- and under-prediction. According to our 4000 simulations, the best simulation generated by MSR projects 83 hits over 131 actual landslide sites while the unstable cells cover only 16% of the studied watershed. By contrast, despite the fact that the best simulation deduced from SR projects 120 hits over 131 actual landslide sites, this high performance is only obtained when unstable cells cover an incredibly high landslide cover (~75%) of the entire watershed exhibiting a significant landslide over-prediction. |
url |
http://www.hydrol-earth-syst-sci.net/10/957/2006/hess-10-957-2006.pdf |
work_keys_str_mv |
AT jchuang optimalestimatorforassessinglandslidemodelperformance AT sjkao optimalestimatorforassessinglandslidemodelperformance |
_version_ |
1725261269278130176 |