Evolution of the Hadoop Platform and Ecosystem for High Energy Physics

Evolution of the Hadoop Platform and Ecosystem for High Energy Physics

The interest in using scalable data processing solutions based on Apache Hadoop ecosystem is constantly growing in the High Energy Physics (HEP) community. This drives the need for increased reliability and availability of the central Hadoop service and underlying infrastructure provided to the comm...

Full description

Bibliographic Details
Main Authors: Baranowski Zbigniew, Kleszcz Emil, Kothuri Prasanth, Canali Luca, Castellotti Riccardo, Martin Marquez Manuel, Matos de Barros Nuno Guilherme, Motesnitsalis Evangelos, Mrowczynski Piotr, Luna Duran Jose Carlos
Format: Article
Language:English
Published: EDP Sciences 2019-01-01
Series:EPJ Web of Conferences
Online Access:https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_04058.pdf
id doaj-b31476b733fa43798f477365f85879ac
record_format Article
spelling doaj-b31476b733fa43798f477365f85879ac2021-08-02T10:08:54ZengEDP SciencesEPJ Web of Conferences2100-014X2019-01-012140405810.1051/epjconf/201921404058epjconf_chep2018_04058Evolution of the Hadoop Platform and Ecosystem for High Energy PhysicsBaranowski ZbigniewKleszcz EmilKothuri PrasanthCanali LucaCastellotti RiccardoMartin Marquez ManuelMatos de Barros Nuno GuilhermeMotesnitsalis EvangelosMrowczynski PiotrLuna Duran Jose CarlosThe interest in using scalable data processing solutions based on Apache Hadoop ecosystem is constantly growing in the High Energy Physics (HEP) community. This drives the need for increased reliability and availability of the central Hadoop service and underlying infrastructure provided to the community by the CERN IT department. This paper reports on the overall status of the Hadoop platform and related Hadoop and Spark service at CERN, detailing recent enhancements and features introduced in many areas including the service configuration, availability, alerting, monitoring and data protection, in order to meet the new requirements posed by the users’ community.https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_04058.pdf
collection DOAJ
language English
format Article
sources DOAJ
author Baranowski Zbigniew
Kleszcz Emil
Kothuri Prasanth
Canali Luca
Castellotti Riccardo
Martin Marquez Manuel
Matos de Barros Nuno Guilherme
Motesnitsalis Evangelos
Mrowczynski Piotr
Luna Duran Jose Carlos
spellingShingle Baranowski Zbigniew
Kleszcz Emil
Kothuri Prasanth
Canali Luca
Castellotti Riccardo
Martin Marquez Manuel
Matos de Barros Nuno Guilherme
Motesnitsalis Evangelos
Mrowczynski Piotr
Luna Duran Jose Carlos
Evolution of the Hadoop Platform and Ecosystem for High Energy Physics
EPJ Web of Conferences
author_facet Baranowski Zbigniew
Kleszcz Emil
Kothuri Prasanth
Canali Luca
Castellotti Riccardo
Martin Marquez Manuel
Matos de Barros Nuno Guilherme
Motesnitsalis Evangelos
Mrowczynski Piotr
Luna Duran Jose Carlos
author_sort Baranowski Zbigniew
title Evolution of the Hadoop Platform and Ecosystem for High Energy Physics
title_short Evolution of the Hadoop Platform and Ecosystem for High Energy Physics
title_full Evolution of the Hadoop Platform and Ecosystem for High Energy Physics
title_fullStr Evolution of the Hadoop Platform and Ecosystem for High Energy Physics
title_full_unstemmed Evolution of the Hadoop Platform and Ecosystem for High Energy Physics
title_sort evolution of the hadoop platform and ecosystem for high energy physics
publisher EDP Sciences
series EPJ Web of Conferences
issn 2100-014X
publishDate 2019-01-01
description The interest in using scalable data processing solutions based on Apache Hadoop ecosystem is constantly growing in the High Energy Physics (HEP) community. This drives the need for increased reliability and availability of the central Hadoop service and underlying infrastructure provided to the community by the CERN IT department. This paper reports on the overall status of the Hadoop platform and related Hadoop and Spark service at CERN, detailing recent enhancements and features introduced in many areas including the service configuration, availability, alerting, monitoring and data protection, in order to meet the new requirements posed by the users’ community.
url https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_04058.pdf
work_keys_str_mv AT baranowskizbigniew evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT kleszczemil evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT kothuriprasanth evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT canaliluca evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT castellottiriccardo evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT martinmarquezmanuel evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT matosdebarrosnunoguilherme evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT motesnitsalisevangelos evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT mrowczynskipiotr evolutionofthehadoopplatformandecosystemforhighenergyphysics
AT lunaduranjosecarlos evolutionofthehadoopplatformandecosystemforhighenergyphysics
_version_ 1721234143915802624

Similar Items