Evaluating non-relational storage technology for HEP metadata and meta-data catalog

M. A. Grigorieva, M. V. Golosova, M. Y. Gubin, A. A. Klimentov, V. V. Osipova, E. A. Ryabinkin

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Large-scale scientific experiments produce vast volumes of data. These data are stored, processed and analyzed in a distributed computing environment. The life cycle of experiment is managed by specialized software like Distributed Data Management and Workload Management Systems. In order to be interpreted and mined, experimental data must be accompanied by auxiliary metadata, which are recorded at each data processing step. Metadata describes scientific data and represent scientific objects or results of scientific experiments, allowing them to be shared by various applications, to be recorded in databases or published via Web. Processing and analysis of constantly growing volume of auxiliary metadata is a challenging task, not simpler than the management and processing of experimental data itself. Furthermore, metadata sources are often loosely coupled and potentially may lead to an end-user inconsistency in combined information queries. To aggregate and synthesize a range of primary metadata sources, and enhance them with flexible schema-less addition of aggregated data, we are developing the Data Knowledge Base architecture serving as the intelligence behind GUIs and APIs.

Original languageEnglish
Article number012017
JournalJournal of Physics: Conference Series
Volume762
Issue number1
DOIs
Publication statusPublished - 21 Nov 2016

Fingerprint

metadata
catalogs
application programming interface
data management
graphical user interface
intelligence
management systems
computer programs
cycles

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Cite this

Evaluating non-relational storage technology for HEP metadata and meta-data catalog. / Grigorieva, M. A.; Golosova, M. V.; Gubin, M. Y.; Klimentov, A. A.; Osipova, V. V.; Ryabinkin, E. A.

In: Journal of Physics: Conference Series, Vol. 762, No. 1, 012017, 21.11.2016.

Research output: Contribution to journalArticle

@article{b749cad17f994154ac4e209b02c7920f,
title = "Evaluating non-relational storage technology for HEP metadata and meta-data catalog",
abstract = "Large-scale scientific experiments produce vast volumes of data. These data are stored, processed and analyzed in a distributed computing environment. The life cycle of experiment is managed by specialized software like Distributed Data Management and Workload Management Systems. In order to be interpreted and mined, experimental data must be accompanied by auxiliary metadata, which are recorded at each data processing step. Metadata describes scientific data and represent scientific objects or results of scientific experiments, allowing them to be shared by various applications, to be recorded in databases or published via Web. Processing and analysis of constantly growing volume of auxiliary metadata is a challenging task, not simpler than the management and processing of experimental data itself. Furthermore, metadata sources are often loosely coupled and potentially may lead to an end-user inconsistency in combined information queries. To aggregate and synthesize a range of primary metadata sources, and enhance them with flexible schema-less addition of aggregated data, we are developing the Data Knowledge Base architecture serving as the intelligence behind GUIs and APIs.",
author = "Grigorieva, {M. A.} and Golosova, {M. V.} and Gubin, {M. Y.} and Klimentov, {A. A.} and Osipova, {V. V.} and Ryabinkin, {E. A.}",
year = "2016",
month = "11",
day = "21",
doi = "10.1088/1742-6596/762/1/012017",
language = "English",
volume = "762",
journal = "Journal of Physics: Conference Series",
issn = "1742-6588",
publisher = "IOP Publishing Ltd.",
number = "1",

}

TY - JOUR

T1 - Evaluating non-relational storage technology for HEP metadata and meta-data catalog

AU - Grigorieva, M. A.

AU - Golosova, M. V.

AU - Gubin, M. Y.

AU - Klimentov, A. A.

AU - Osipova, V. V.

AU - Ryabinkin, E. A.

PY - 2016/11/21

Y1 - 2016/11/21

N2 - Large-scale scientific experiments produce vast volumes of data. These data are stored, processed and analyzed in a distributed computing environment. The life cycle of experiment is managed by specialized software like Distributed Data Management and Workload Management Systems. In order to be interpreted and mined, experimental data must be accompanied by auxiliary metadata, which are recorded at each data processing step. Metadata describes scientific data and represent scientific objects or results of scientific experiments, allowing them to be shared by various applications, to be recorded in databases or published via Web. Processing and analysis of constantly growing volume of auxiliary metadata is a challenging task, not simpler than the management and processing of experimental data itself. Furthermore, metadata sources are often loosely coupled and potentially may lead to an end-user inconsistency in combined information queries. To aggregate and synthesize a range of primary metadata sources, and enhance them with flexible schema-less addition of aggregated data, we are developing the Data Knowledge Base architecture serving as the intelligence behind GUIs and APIs.

AB - Large-scale scientific experiments produce vast volumes of data. These data are stored, processed and analyzed in a distributed computing environment. The life cycle of experiment is managed by specialized software like Distributed Data Management and Workload Management Systems. In order to be interpreted and mined, experimental data must be accompanied by auxiliary metadata, which are recorded at each data processing step. Metadata describes scientific data and represent scientific objects or results of scientific experiments, allowing them to be shared by various applications, to be recorded in databases or published via Web. Processing and analysis of constantly growing volume of auxiliary metadata is a challenging task, not simpler than the management and processing of experimental data itself. Furthermore, metadata sources are often loosely coupled and potentially may lead to an end-user inconsistency in combined information queries. To aggregate and synthesize a range of primary metadata sources, and enhance them with flexible schema-less addition of aggregated data, we are developing the Data Knowledge Base architecture serving as the intelligence behind GUIs and APIs.

UR - http://www.scopus.com/inward/record.url?scp=85002245024&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85002245024&partnerID=8YFLogxK

U2 - 10.1088/1742-6596/762/1/012017

DO - 10.1088/1742-6596/762/1/012017

M3 - Article

VL - 762

JO - Journal of Physics: Conference Series

JF - Journal of Physics: Conference Series

SN - 1742-6588

IS - 1

M1 - 012017

ER -