MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase
Meteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of observational data are generated gradually by technologies such as geographical remote sensing, meteorological radar satellite, etc. that ma...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2016-03-01
|
Series: | Future Internet |
Subjects: | |
Online Access: | http://www.mdpi.com/1999-5903/8/1/6 |
id |
doaj-81a635abbad34874acac357cbb572a97 |
---|---|
record_format |
Article |
spelling |
doaj-81a635abbad34874acac357cbb572a972020-11-24T22:26:04ZengMDPI AGFuture Internet1999-59032016-03-0181610.3390/fi8010006fi8010006MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBaseTinghuai Ma0Xichao Xu1Meili Tang2Yuanfeng Jin3Wenhai Shen4Jiangsu Engineering Centre of Network Monitoring, Nanjing University of information Science and Technology, Nanjing 210044, Jiangsu, ChinaSchool of Computer Software, Nanjing University of Information Science & Technology, Nanjing 210044, Jiangsu, ChinaSchool of Public Administration, Nanjing University of Information Science & Technology, Nanjing 210044, Jiangsu, ChinaDepartmet of Mathmatics, YanBian University, Yanji 133002, Jilin, ChinaNational Meteorological Information Center, Beijing 100080, ChinaMeteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of observational data are generated gradually by technologies such as geographical remote sensing, meteorological radar satellite, etc. that makes data analysis in weather forecasting more precise but also poses a threat to the traditional method of data storage. In this paper, we present MHBase, (Meteorological data based on HBase (Hadoop Database), a distributed real-time query scheme for meteorological data based on HBase. The calibrated data obtained from terminal devices will be partitioned into HBase and persisted to HDFS (the Hadoop Distributed File System). We propose two algorithms (the Indexed Store and the Indexed Retrieve Algorithms) to implement a secondary index using HBase Coprocessors, which allow MHbase to provide high performance data querying on columns other than rowkey. Experimental results show that the performance of MHBase can satisfy the basic demands of meteorological business services.http://www.mdpi.com/1999-5903/8/1/6meteorological datadistributed storageHBasesecondary index |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Tinghuai Ma Xichao Xu Meili Tang Yuanfeng Jin Wenhai Shen |
spellingShingle |
Tinghuai Ma Xichao Xu Meili Tang Yuanfeng Jin Wenhai Shen MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase Future Internet meteorological data distributed storage HBase secondary index |
author_facet |
Tinghuai Ma Xichao Xu Meili Tang Yuanfeng Jin Wenhai Shen |
author_sort |
Tinghuai Ma |
title |
MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase |
title_short |
MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase |
title_full |
MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase |
title_fullStr |
MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase |
title_full_unstemmed |
MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase |
title_sort |
mhbase: a distributed real-time query scheme for meteorological data based on hbase |
publisher |
MDPI AG |
series |
Future Internet |
issn |
1999-5903 |
publishDate |
2016-03-01 |
description |
Meteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of observational data are generated gradually by technologies such as geographical remote sensing, meteorological radar satellite, etc. that makes data analysis in weather forecasting more precise but also poses a threat to the traditional method of data storage. In this paper, we present MHBase, (Meteorological data based on HBase (Hadoop Database), a distributed real-time query scheme for meteorological data based on HBase. The calibrated data obtained from terminal devices will be partitioned into HBase and persisted to HDFS (the Hadoop Distributed File System). We propose two algorithms (the Indexed Store and the Indexed Retrieve Algorithms) to implement a secondary index using HBase Coprocessors, which allow MHbase to provide high performance data querying on columns other than rowkey. Experimental results show that the performance of MHBase can satisfy the basic demands of meteorological business services. |
topic |
meteorological data distributed storage HBase secondary index |
url |
http://www.mdpi.com/1999-5903/8/1/6 |
work_keys_str_mv |
AT tinghuaima mhbaseadistributedrealtimequeryschemeformeteorologicaldatabasedonhbase AT xichaoxu mhbaseadistributedrealtimequeryschemeformeteorologicaldatabasedonhbase AT meilitang mhbaseadistributedrealtimequeryschemeformeteorologicaldatabasedonhbase AT yuanfengjin mhbaseadistributedrealtimequeryschemeformeteorologicaldatabasedonhbase AT wenhaishen mhbaseadistributedrealtimequeryschemeformeteorologicaldatabasedonhbase |
_version_ |
1725754844877160448 |