CERN Tape Archive: production status, migration from CASTOR and new features
During 2019 and 2020, the CERN tape archive (CTA) will receive new data from LHC experiments and import existing data from CASTOR, which will be phased out for LHC experiments before Run 3. This contribution will present the statuses of CTA as a service and of its integration with EOS and FTS and th...
Main Authors: | , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
EDP Sciences
2020-01-01
|
Series: | EPJ Web of Conferences |
Online Access: | https://www.epj-conferences.org/articles/epjconf/pdf/2020/21/epjconf_chep2020_04013.pdf |
id |
doaj-0559b759816e4a1e8050f7d4003fb32a |
---|---|
record_format |
Article |
spelling |
doaj-0559b759816e4a1e8050f7d4003fb32a2021-08-02T22:55:36ZengEDP SciencesEPJ Web of Conferences2100-014X2020-01-012450401310.1051/epjconf/202024504013epjconf_chep2020_04013CERN Tape Archive: production status, migration from CASTOR and new featuresCano Eric0Bahyl Vladimír1Caffy Cédric2Cancio Germán3Davis Michael4Kotlyar Viktor5Leduc Julien6Lin Tao7Murray Steven8CERN—European Organization for Nuclear ResearchCERN—European Organization for Nuclear ResearchCERN—European Organization for Nuclear ResearchCERN—European Organization for Nuclear ResearchCERN—European Organization for Nuclear ResearchInstitute for High Energy Physics named by A.A. Logunov of National Research Center “Kurchatov Institute”CERN—European Organization for Nuclear ResearchInstitute of High Energy Physics, Chinese Academy of SciencesCERN—European Organization for Nuclear ResearchDuring 2019 and 2020, the CERN tape archive (CTA) will receive new data from LHC experiments and import existing data from CASTOR, which will be phased out for LHC experiments before Run 3. This contribution will present the statuses of CTA as a service and of its integration with EOS and FTS and the data flow chains of LHC experiments. The latest enhancements and additions to the software as well as the development outlook will be presented. With the development of the repack function, a necessary behind-the-scenes feature, CTA can now take over custodial data and handle media migration, compaction and failures. Further metadata handling optimisations allowed doubling the maximum file rate performance to 200Hz per queue. New retrieve scheduling options are being developed at the request of experiments, with optional FIFO behaviour to ensure better control of the timing for datasets retrieve, and fair share support for competing activities within the same VO. Support for multiple backend databases (Oracle, PostgreSQL, MySQL) have been developed at CERN and contributed by external institutes. This contribution will also report on the challenges of and solutions for migrating data from the decades old CASTOR to CTA. The practical example of the preparation for the migration of ATLAS data will be presented.https://www.epj-conferences.org/articles/epjconf/pdf/2020/21/epjconf_chep2020_04013.pdf |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Cano Eric Bahyl Vladimír Caffy Cédric Cancio Germán Davis Michael Kotlyar Viktor Leduc Julien Lin Tao Murray Steven |
spellingShingle |
Cano Eric Bahyl Vladimír Caffy Cédric Cancio Germán Davis Michael Kotlyar Viktor Leduc Julien Lin Tao Murray Steven CERN Tape Archive: production status, migration from CASTOR and new features EPJ Web of Conferences |
author_facet |
Cano Eric Bahyl Vladimír Caffy Cédric Cancio Germán Davis Michael Kotlyar Viktor Leduc Julien Lin Tao Murray Steven |
author_sort |
Cano Eric |
title |
CERN Tape Archive: production status, migration from CASTOR and new features |
title_short |
CERN Tape Archive: production status, migration from CASTOR and new features |
title_full |
CERN Tape Archive: production status, migration from CASTOR and new features |
title_fullStr |
CERN Tape Archive: production status, migration from CASTOR and new features |
title_full_unstemmed |
CERN Tape Archive: production status, migration from CASTOR and new features |
title_sort |
cern tape archive: production status, migration from castor and new features |
publisher |
EDP Sciences |
series |
EPJ Web of Conferences |
issn |
2100-014X |
publishDate |
2020-01-01 |
description |
During 2019 and 2020, the CERN tape archive (CTA) will receive new data from LHC experiments and import existing data from CASTOR, which will be phased out for LHC experiments before Run 3.
This contribution will present the statuses of CTA as a service and of its integration with EOS and FTS and the data flow chains of LHC experiments.
The latest enhancements and additions to the software as well as the development outlook will be presented. With the development of the repack function, a necessary behind-the-scenes feature, CTA can now take over custodial data and handle media migration, compaction and failures. Further metadata handling optimisations allowed doubling the maximum file rate performance to 200Hz per queue.
New retrieve scheduling options are being developed at the request of experiments, with optional FIFO behaviour to ensure better control of the timing for datasets retrieve, and fair share support for competing activities within the same VO.
Support for multiple backend databases (Oracle, PostgreSQL, MySQL) have been developed at CERN and contributed by external institutes.
This contribution will also report on the challenges of and solutions for migrating data from the decades old CASTOR to CTA. The practical example of the preparation for the migration of ATLAS data will be presented. |
url |
https://www.epj-conferences.org/articles/epjconf/pdf/2020/21/epjconf_chep2020_04013.pdf |
work_keys_str_mv |
AT canoeric cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT bahylvladimir cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT caffycedric cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT canciogerman cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT davismichael cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT kotlyarviktor cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT leducjulien cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT lintao cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures AT murraysteven cerntapearchiveproductionstatusmigrationfromcastorandnewfeatures |
_version_ |
1721225932197330944 |