Distributed OAIS-Based digital preservation system with HDFS technology

The paper describes architecture of a distributed OAIS-based digital preservation system which uses HDFS as a file storage system and supports wide distribution on a number of cluster's nodes. It is based on Apache Hadoop framework - a reliable open source solution with well horizontally scalab...

Full description

Bibliographic Details
Main Authors: Nikita Voinov, Pavel Drobintsev, Vsevolod Kotlyarov, Igor Nikiforov
Format: Article
Language:English
Published: FRUCT 2017-04-01
Series:Proceedings of the XXth Conference of Open Innovations Association FRUCT
Subjects:
Online Access:https://fruct.org/publications/fruct20/files/Voi.pdf
id doaj-333205cf53874b83a2d941f5fca8c9dc
record_format Article
spelling doaj-333205cf53874b83a2d941f5fca8c9dc2020-11-24T21:08:15ZengFRUCTProceedings of the XXth Conference of Open Innovations Association FRUCT2305-72542343-07372017-04-017762049149710.23919/FRUCT.2017.8071353Distributed OAIS-Based digital preservation system with HDFS technologyNikita Voinov0Pavel Drobintsev1Vsevolod Kotlyarov2Igor Nikiforov3Peter the Great Saint-Petersburg Polytechnic University, Saint-Petersburg, RussiaPeter the Great Saint-Petersburg Polytechnic University, Saint-Petersburg, RussiaPeter the Great Saint-Petersburg Polytechnic University, Saint-Petersburg, RussiaDell EMC/Peter the Great Saint-Petersburg Polytechnic University, Saint-Petersburg, RussiaThe paper describes architecture of a distributed OAIS-based digital preservation system which uses HDFS as a file storage system and supports wide distribution on a number of cluster's nodes. It is based on Apache Hadoop framework - a reliable open source solution with well horizontally scalable distributed architecture. Novelty of the proposed system is defined by the fact that none of existing OAIS digital preservation systems use HDFS storage for both structured and unstructured data archiving. Implementation of the system's prototype and results of its testing are also shown.https://fruct.org/publications/fruct20/files/Voi.pdf digital preservationdistributed systemhorizontal scalabilityOAISApache HadoopHDFS
collection DOAJ
language English
format Article
sources DOAJ
author Nikita Voinov
Pavel Drobintsev
Vsevolod Kotlyarov
Igor Nikiforov
spellingShingle Nikita Voinov
Pavel Drobintsev
Vsevolod Kotlyarov
Igor Nikiforov
Distributed OAIS-Based digital preservation system with HDFS technology
Proceedings of the XXth Conference of Open Innovations Association FRUCT
digital preservation
distributed system
horizontal scalability
OAIS
Apache Hadoop
HDFS
author_facet Nikita Voinov
Pavel Drobintsev
Vsevolod Kotlyarov
Igor Nikiforov
author_sort Nikita Voinov
title Distributed OAIS-Based digital preservation system with HDFS technology
title_short Distributed OAIS-Based digital preservation system with HDFS technology
title_full Distributed OAIS-Based digital preservation system with HDFS technology
title_fullStr Distributed OAIS-Based digital preservation system with HDFS technology
title_full_unstemmed Distributed OAIS-Based digital preservation system with HDFS technology
title_sort distributed oais-based digital preservation system with hdfs technology
publisher FRUCT
series Proceedings of the XXth Conference of Open Innovations Association FRUCT
issn 2305-7254
2343-0737
publishDate 2017-04-01
description The paper describes architecture of a distributed OAIS-based digital preservation system which uses HDFS as a file storage system and supports wide distribution on a number of cluster's nodes. It is based on Apache Hadoop framework - a reliable open source solution with well horizontally scalable distributed architecture. Novelty of the proposed system is defined by the fact that none of existing OAIS digital preservation systems use HDFS storage for both structured and unstructured data archiving. Implementation of the system's prototype and results of its testing are also shown.
topic digital preservation
distributed system
horizontal scalability
OAIS
Apache Hadoop
HDFS
url https://fruct.org/publications/fruct20/files/Voi.pdf
work_keys_str_mv AT nikitavoinov distributedoaisbaseddigitalpreservationsystemwithhdfstechnology
AT paveldrobintsev distributedoaisbaseddigitalpreservationsystemwithhdfstechnology
AT vsevolodkotlyarov distributedoaisbaseddigitalpreservationsystemwithhdfstechnology
AT igornikiforov distributedoaisbaseddigitalpreservationsystemwithhdfstechnology
_version_ 1716760324651089920